Diverse sources of reward value signals in the basal ganglia nuclei transmitted to the lateral habenula in the monkey

Abstract

The lateral habenula (LHb) plays an important role in motivational decision making. Neurons in the primate LHb signal negative ‘reward prediction errors’ and inhibit midbrain dopamine (DA) neurons. These negative reward prediction error signals in the LHb are, at least partly, provided by a distinct group of neurons in the border region of the globus pallidus internal segment (GPb). However, it is still unclear whether other basal ganglia nuclei provide the LHb with reward signals, either through the GPb or through different circuits. As a first step to answer this question, we electrically stimulated various parts of the basal ganglia and monitored the neural activity in the LHb in the awake monkey. First, we found that low intensity stimulations in the GPb and the internal segment of the globus pallidus (GPi) evoked a short latency (5 ms) excitatory response in LHb neurons. Second, LHb neurons were inhibited by stimulations in the ventral pallidum (VP). These results suggest that reward-related signals are transmitted to the LHb mainly through excitatory connections from the GPb and inhibitory connections from the VP. Finally, excitations or inhibitions are induced in LHb neurons from diverse but patchy regions in the striatum. These effects have considerably longer latencies, suggesting that they may be mediated by the GPb or the VP. The patchy nature of the stimulation effect raises the possibility that the striosomes are the source of reward-related signals transmitted to the LHb.

Introduction

The lateral habenula (LHb) is involved in fundamental aspects of animal behavior (Hikosaka, 2010). In particular, its role in motivational and emotional processes, as well as their disorders, has been a focus of recent research (e.g., Matsumoto and Hikosaka, 2007; Brown et al., 2010; Sartorius et al., 2010; James et al., 2011; Li et al., 2011; Shabel et al., 2012; Stamatakis and Stuber, 2012). Neurons in the primate lateral LHb are excited by visual stimuli that predict the absence of reward and are inhibited by stimuli that predict the presence of reward (Matsumoto and Hikosaka, 2007), and inhibit dopamine (DA) neurons in the midbrain (Lisoprawski et al., 1980; Christoph et al., 1986; Ji and Shepard, 2007; Matsumoto and Hikosaka, 2007), mainly through the rostromedial tegmental nucleus (RMTg; Jhou et al., 2009; Hong et al., 2011; Barrot et al., 2012).

In our previous study (Hong and Hikosaka, 2008a) we showed that, (1) neurons in the border region of the globus pallidus internal segment (GPb) projected their axons to the LHb, (2) most of the LHb-projecting GPb neurons encoded negative reward prediction errors similar to LHb neurons, and (3) their reward-related activity started earlier than the activity of LHb neurons. These results suggest that the GPb-LHb connection is excitatory, although a minority of GPb neurons encoding positive reward prediction errors may have inhibitory connections to the LHb (Hong and Hikosaka, 2008a). This hypothesis was supported by studies using rats (Shabel et al., 2012) and lampreys (Stephenson-Jones et al., 2013).

How then does the GPb-LHb circuit acquire the signals that are necessary to create reward prediction error signals? One candidate for the source of such signals is the striatum. Physiologically, it is known that many striatal neurons show sensory responses that are modulated by expected reward values (Kawagoe et al., 1998; Oyama et al., 2010; Ambroggi et al., 2011), or exhibit sustained activity predicting sensory stimuli, actions, or reward itself (Hikosaka et al., 1989; Tremblay et al., 1998; Lauwereyns et al., 2002; Lau and Glimcher, 2008; Hori et al., 2009). Anatomically, some studies have suggested that the striatum sends signals to the LHb through indirect connections. In rats, Rajakumar et al. (1993) reported that the rostral part of the entopeduncular nucleus (EPN), which showed strong connections to the LHb, received projections from the striatum, particularly from the striosome (Graybiel and Ragsdale, 1978). In monkeys, using a manganese tracer method, Saleem et al. (2002) found that the striatum had indirect but strong projections to the LHb. However, it was unknown whether the striataum-LHb connection was mediated by the GPb. There may be other routes for the striatum to influence the LHb.

To examine the hypothesis that the striatum is the origin of the input to the LHb, we electrically stimulated various parts of the basal ganglia while recording from the LHb. The spatial distribution of the effective stimulation sites as well as the latency and the direction of modulation of the LHb activity provide suggestions for the functional circuits connecting the basal ganglia to the LHb. An initial result of this study was presented in the SfN abstract (Hong and Hikosaka, 2008b).

Materials and Methods

Three rhesus monkeys (Macaca mulatta), B, C and D, were used as subjects in this study. All animal care and experimental procedures were approved by the National Eye Institute and Institutional Animal Care and Use Committee and complied with the Public Health Service Policy on the humane care and use of laboratory animals.

Electrophysiology

One recording chamber was placed over the midline of the occipital cortex, tilted posteriorly by 40°, and was aimed at the LHb; another chamber for the stimulating electrode was placed over the frontoparietal cortex, tilted laterally by 35°, and was aimed at the GPi. Recordings and electrical stimulations were performed using tungsten electrodes (Frederick Haer) that were advanced by an oil-driven micromanipulator (MO-97A, Narishige). The recording and stimulation sites were determined using a grid system, which allowed recordings at every 1 mm between penetrations. An electrode was introduced into the brain through a stainless steel guide tube, which was inserted into one of the grid holes and then to the brain via the dura. We preferentially recorded multi-unit activity in the LHb using electrodes with low impedances (about 0.5 M Ohm). The signal was amplified with a band-pass filter (200 Hz–5 kHz; BAK, Mount Airy, MD) and collected at 40 kHz via a custom-made window discriminator (MEX). Single neurons were isolated online using a custom voltage-time window discrimination software (MEX, LSR/NEI/NIH).

To record LHb activity, the position of the LHb was mapped first by MRI. The electrophysiological features of the LHb (Matsumoto and Hikosaka, 2007) were also used to locate the LHb. To examine the basal ganglia’s influence on the LHb, electric stimulations were applied to the basal ganglia while recording multi-unit activity from the LHb (Figure 1). In most cases we used 1 biphasic pulse of 100 μA lasting 0.2 ms. We chose these parameters because it was shown previously that stimulation with similar parameters in the CN in awake monkeys produced significant effects on single neurons in the substantia nigra pars reticulata (Hikosaka et al., 1993). However, single pulse stimulation in the striatum often elicited only weak effects on LHb neurons. For this reason, we used three biphasic pulses of 133 μA each lasting 0.2 ms at 300 Hz (for the striatum only). The inter-stimulation interval was about 1 sec with some variability. The stimulation-triggered LHb activity was stored and analyzed offline. To see the effect of stimulation at different structures of the basal ganglia, stimulations were applied 100 times at every 200 μm along the electrode penetration. To minimize stimulation-induced electrical artifacts, we used a commercially available artifact remover (Artifact Zapper-1, Riverband Instrument).

Figure 1.

To examine the influence of the basal ganglia on the LHb, electrical stimulation was applied in the basal ganglia (left) while recording multiunit activity from the LHb (right). Stimulation was applied every 200 μm along the track of the stimulating electrode (left), 100 times at each depth. The stimulation was either 1 biphasic pulse of 100 μA (for all structures of basal ganglia) or 3 biphasic pulses of 133 μA each (only for striatum).

Data analysis

At the end of each penetration, a map of effective stimulation sites was made by averaging the 100 times of stimulating effect at every 200 μm along the whole tract. At each depth of stimulation, LHb neuron activity during the 200 ms period preceding the stimulation onset was analyzed to generate a 95% confidence threshold. The threshold was used to determine the significant deviation of LHb activity after stimulation. In the center figures of Figures 2 and 3, significant excitations are shown in white and significant inhibitions are shown in black.

Figure 2.

Effects of electrical stimulation along electrode tracks through the putamen-pallidal region. (A) Stimulations along the putamen-globus pallidus external segment (GPe)-GPi track. Left: The location of the stimulating electrode track. The red line indicates where the stimulation data were collected. Center: Significant effects (white: excitation, black: inhibition) shown in depth (ordinate) and time (abscissa). At each depth of stimulation, the LHb neuron activity in a pre-stimulation period (duration: 200 ms) was analyzed to generate a 95% confidence threshold. This threshold was used to determine a significant post-stimulation deviation of the LHb activity. The anatomical landmarks are indicated on the left side of the figure (WM: white matter; Cl: claustrum). The vertical blue bar indicates 1 mm. Right: Average LHb activity (baseline subtracted PSTH) in the structure corresponding to the bracket next to each peri-stimulus time histogram (PSTH). (B) Stimulations along the putamen-VP track. The same convention as in (A).

Figure 3.

Effects of stimulation along the electrode tracks through the striatum. Electric stimulations of 1 biphasic pulse of 100 μA were used in (A). Three pulse stimulations (3 biphasic pulses, 133 μA each) were used for (B) and (C). The same convention as in Figure 2.

To determine the latency of the stimulation-induced response of LHb neurons, we used a Poisson distribution test (Hong et al., 2011). For this, we first counted the number of accumulated spikes across the trials within a 1 ms bin along the 500 ms period before stimulation. Using these data, a histogram was constructed, with the abscissa representing the number of spikes and the ordinate representing the number of bins that had the number of spikes corresponding to the values on the abscissa. The histogram was fitted with a Poisson distribution curve. Using the Poisson curve, the threshold value of spikes per bin was determined that matched the p value of 0.01. Then, the number of spikes of each bin during the post-stimulation period was examined. The point in time when the histogram exceeded the significance level was taken to be the time of significant modulation by the stimulation.

Results

To examine the basal ganglia’s influence on the LHb, electric stimulation was applied in the basal ganglia while recording multi-unit activity from the LHb. Across the experiments, the stimulating electrode was aimed at different positions of the striatum, while the recording electrode was aimed at the LHb on the same side. We monitored multi-unit activity, rather than single-unit activity, for two reasons. First, LHb neurons show very similar activity in relation to changes in reward values (Matsumoto and Hikosaka, 2007). Second, multi-unit activity remained more stable even when the recording lasted several hours.

Figure 2A shows the effects of electrical stimulations (one bi-phasic 100 μA pulse of 0.2 ms duration) along the electrode track traveling through the putamen, GPe, and GPi. The stimulation effect changed dramatically with the depth of the stimulating electrode. Stimulations in the putamen induced little effect. As the stimulating position moved closer to the border part between the putamen and GPe, a small excitatory effect appeared about 35 ms after the stimulation. As the stimulating electrode approached the GPe, inhibitory and then excitatory effects appeared with shorter latencies (about 25 ms). In the dorsal part of the GPe, however, there was little stimulation effect. In the ventral GPe, an excitatory effect appeared at about 20 ms. Interestingly, as soon as the electrode came out of the GPe and entered the GPe-GPi border region (GPb), a strong phasic excitatory response occurred with a short latency (5 ms), which was followed by an inhibition. The phasic excitation continued throughout the GPi and further into a region ventral to the GPi (which may also belong to the GPb). The data suggest that LHb neurons receive excitatory or inhibitory inputs from different regions in the basal ganglia and that their sources were distributed in a patchy manner.

Another source of short latency responses was found in the VP (Figure 2B). The stimulating electrode passed along the lateral edge of the putamen, the anterior part of the GPe, the anterior commissure (AC), and reached the VP. The stimulation effect was mostly confined in the VP and was an inhibitory response that started about 15 ms after the stimulation and lasted about 40 ms.

Stimulation in the striatum elicited diverse effects in the multi-unit LHb activity (Figure 3). Excitatory responses were evoked from patchy areas in the putamen, in its rostral portion (Figure 3A) and central portion (Figure 3B) (latencies: 30–40 ms). Inhibitory responses with shorter latencies (15–25 ms) were evoked from the ventral portion of the central putamen, close to the transition to the GPe (Figures 2A, 3B). Stimulations in the ventral striatum, including the nucleus accumbens (NAc), evoked inhibitory responses (latencies: 50–70 ms) followed by an excitation (Figure 3A, center). Stimulations in the rostral portion of the caudate nucleus (CN) evoked excitatory responses with different latencies (dorsal CN: 60 ms, ventral CN: 30–40 ms). Overall, the effective sites of stimulation were distributed in a patchy manner in the striatum. Their effects (i.e., combination of excitatory and inhibitory effects) and latencies were variable across the patches.

Discussion

GPb-LHb connection

The data in Figure 2A suggest that LHb neurons receive excitatory and inhibitory inputs from different regions in the basal ganglia. The short latency response evoked from the GPb-GPi region may be explained by the connection from the GPb to the LHb (Parent et al., 1981). We previously found that many GPb neurons projected their axons to the LHb and a majority of them showed negative reward modulations of their responses to visual saccade targets (Hong and Hikosaka, 2008a). Since LHb neurons also showed similar negative reward modulations, the GPb-LHb connection was considered to be excitatory. Our present data (Figure 2A) are consistent with this hypothesis. The excitatory nature of the GPb-LHb connection was supported by recent studies in rats (Barroso-Chinea et al., 2008; Shabel et al., 2012) and lampreys (Stephenson-Jones et al., 2013).

The results in Figure 2A reflect the functional anatomy reported in the literature. The short latency excitatory effect was observed not only from the GPb but also from the underlying areas including the GPi and an area below it. This may be due to the anatomical structure of the GPb. Neurons that project to the LHb are located not only in the GPb but also in the middle of the GPi (close to the accessory medullary lamina, AML) and, occasionally, below the border area of the GPi (Hong and Hikosaka, 2008a). If the GPb is defined as the area that connects to the LHb, it has a complex 3D layered structure (see also Parent et al., 2001).We also speculate that the axons of the GPb neurons travel across these layers (i.e., top layer: GPe-GPi border, middle layer: AML, bottom layer: ventral to GPi) before heading toward the LHb. Stimulation at the areas ventromedial to the GPe-GPi border would activate efferent axons of GPb neurons and induce strong effects on LHb neurons.

VP-LHb connection

The VP was another structure from which short latency effects were evoked in LHb neurons (Figure 2B). Anatomically, the VP is known to project to the LHb among several other areas (Haber and Knutson, 2010). Berridge and colleagues have suggested that the VP may serve as a ‘limbic final common pathway’ for processing of reward (Smith et al., 2009), a conclusion based on studies using various methods: lesions (Cromwell and Berridge, 1993), inactivations (Farrar et al., 2008), chemical manipulations (Stratford et al., 1999; Shimura et al., 2006), neuronal recordings (Wheeler and Carelli, 2006; Tindell et al., 2006). Human imaging studies are largely consistent with this conclusion (Simmons et al., 2013). A recent study from our lab (Tachibana and Hikosaka, 2012) showed that primate VP neurons encoded expected reward values and costs by changing the level of tonic activity in a stepwise manner. Importantly, a majority of VP neurons were excited by the expectation of a large reward (i.e., positive value coding). Since some VP neurons projecting to the mediodorsal thalamus (Zahm et al., 1987) and the substantia nigra (Bevan et al., 1996) are GABAergic and inhibitory, the VP-LHb connection may also be inhibitory. If so, their positive value coding would be converted to negative value coding in LHb neurons, which is consistent with how LHb neurons behave (Matsumoto and Hikosaka, 2007). However, there is a crucial difference in information processing between VP neurons and LHb neurons: VP neurons represent the expected reward value per se, whereas LHb neurons represent a difference between the actual and expected rewards (i.e., reward prediction error). How the conversion of the information occurs via the VP-LHb connection remains to be solved.

Striatum-LHb connections

We found that stimulations in the striatum induced diverse responses in LHb neurons. The effect of striatal stimulation was excitatory or inhibitory, depending on where the stimulation was applied in the striatum (Figures 2, 3). Inhibitory effects were evoked from the ventral part of the putamen (Figure 3B) and the ventral striatum (Figure 3A). Excitatory effects were evoked from the dorsal part of the putamen (Figures 3A, B) and the CN (Figure 3C). How can we explain these different effects?

The latencies of these effects are longer than those from the GPb or VP, consistent with anatomical findings indicating the striatum-LHb connection is indirect (Saleem et al., 2002). It is conceivable that the effects were mediated by the GPb (Rajakumar et al., 1993) or the VP (Haber et al., 1993; Figure 4). Whether the net effect is excitatory or inhibitory would thus depend on the nature of the disynaptic (or multi-synaptic) connections.

Figure 4.

Striatum-LHb connections: possible circuits through GPb or VP. Our results suggest that the GPb-LHb connection is excitatory while the VP-LHb connection is inhibitory. A majority of GPb and LHb neurons encode negative reward prediction errors (e.g., excited by the unpredicted omission of reward), whereas a majority of VP neurons encode positive reward values (e.g., excited by the prediction of reward), The excitatory-inhibitory nature is unknown for the striatum-GPb and the striatum-VP connections; the sources of these connections might be the striosomes.

Medium spiny neurons comprise the only class of output neurons in the striatum and are GABAergic inhibitory neurons (Tepper and Bolam, 2004). It is then expected that striatal stimulations induce inhibitory responses in GPb and VP neurons. Indeed, Tremblay and Filion (1989) found in the monkey that the ordinary types of GPe and GPi neurons were first inhibited by striatal stimulations. On the other hand, what they called ‘border neurons’ were often (but not always) excited by striatal stimulations, possibly because the signals were mediated through inhibitory axon collaterals within the striatum (Tremblay and Filion, 1989) or GPe (Sadek et al., 2007). The border neurons were distributed in the border regions that surround the GPi as well as the GPe, and showed tonic firing whose frequency was lower than the common types of GPi or GPe neurons (DeLong, 1971). Some of the border neurons may correspond to GPb neurons projecting the LHb (Parent et al., 2001). Assuming that the GPb-LHb connection is excitatory (Hong and Hikosaka, 2008a; Shabel et al., 2012; Stephenson-Jones et al., 2013), the net effect of the striatal stimulation would be excitatory (if GPb neurons are excited) or inhibitory (if GPb neurons are inhibited). Therefore, both the excitatory and inhibitory effects caused by striatal stimulations (Figures 2, 3) could be mediated by GPb neurons.

Compared with the striatum-GPb-LHb circuit, the significance of the striatum-VP-LHb circuit is less clear. Anatomically, the VP is known to receive inputs mainly from the ventral portion of the striatum (Heimer et al., 1987; Usuda et al., 1998) and projects to the LHb (Haber et al., 1993). Although it is commonly thought that the striatum-VP connection is GABAergic and inhibitory (Walaas and Fonnum, 1979), it might not apply to VP neurons that project to the LHb. Therefore, it is unclear which part of the striatum modulates the activity of LHb neurons through the VP or through the GPb.

A remarkable feature of the effects of the striatum stimulation, regardless of the underlying synaptic mechanisms, is that the effective sites are distributed in a patchy manner. It has long been known that the striatum is composed of two chemically defined regions, forming patchy structures (called “striosome”) and surrounding areas (called “matrix”) (Graybiel and Ragsdale, 1978; Gerfen, 1984). We speculate that the patchy effective sites in the striatum (Figure 3) may correspond to striosomes, as suggested by Rajakumar et al. (1993). Striosomes receive inputs from limbic areas, such as the amygdala (Gerfen, 1984; Ragsdale and Graybiel, 1988) and the orbitofrontal-insular cortices (Eblen and Graybiel, 1995), suggesting that they carry emotional or motivational signals. Consistent with this idea, rats quickly learn to self-stimulate the striosomal regions, but not matrix regions (White and Hiroi, 1998). Considering this reinforcing effect of the striosome, and the patchy distribution of the striatal stimulation effects on LHb neurons described above, we speculate that the strisome-GPb-LHb circuit contributes to the formation of negative reward prediction error signals in the LHb.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.