Appetitive and Aversive Information Coding in the Primate Dorsal Raphé Nucleus

Abstract

Serotonin is known to play a key role in the regulation of emotional behavior. There have been conflicting hypotheses about whether the central serotonergic system is involved in positive or negative emotional information processing. To reveal whether and how such opposing information processing can be achieved by single neurons in the dorsal raphé nucleus (DRN), the major source of serotonin in the forebrain, we recorded the activity of DRN neurons while monkeys were conditioned in a Pavlovian procedure with two distinct contexts: an appetitive block where a reward was available; and an aversive one where an airpuff was delivered. We found that single DRN neurons were involved in several aspects of both appetitive and aversive information processing. First, more than half of the recorded DRN neurons discriminated between appetitive and aversive contexts by tonic changes in their activity. In the appetitive context, they then kept track of the expected reward value indicated by the conditioned stimuli. Some of them also encoded an error between the obtained and expected values. In the aversive context, the same neurons maintained tonic modulation in their activity throughout the block. However, modulation of their responses to aversive task events depending on airpuff probability was less common. Together, these results indicate that single DRN neurons encode both appetitive and aversive information, but over differing time scales: relatively shorter for appetitive, and longer for aversive. Such temporally distinct processes of value coding in the DRN may provide the neural basis of emotional information processing in different contexts.

Introduction

Making a decision requires the brain to compute values associated with different contexts, stimuli, and outcomes in appetitive and aversive terms. Dopamine and serotonin appear to be the major neurotransmitters in this process. The role of the dopaminergic system has been studied extensively, and it is implicated in signaling a “reward prediction error” (RPE), which is the difference between the expected reward, called the “state value,” and the actual reward (Mirenowicz and Schultz, 1994; Montague et al., 1996).

In contrast, the role of serotonin remains more elusive. One of the leading hypotheses is that the serotoninergic system is involved in aversive information processing. Aversive cues or outcomes enhance c-Fos expression (Pezzone et al., 1993; Grahn et al., 1999; Takase et al., 2004), neuronal activity (Schweimer and Ungless, 2010), and serotonin levels (Kirby et al., 1995) in the dorsal raphé nucleus (DRN), the major source of serotonin in the forebrain. Serotonin depletion enhances motivational behavior in rats (Fletcher et al., 1999). The inhibition of serotonergic neurons also leads to dopamine-dependent rewarding effects (Liu and Ikemoto, 2007). Thus, the serotonergic system has been thought to encode aversive information and have an opponent influence on the reward circuitry. The function of the serotonergic system in aversive information processing has also been reported in humans; systemic reduction in serotonin levels led to exaggerated aversive processing and deficient response inhibition (for review, Cools et al., 2008).

However, some electrophysiological studies in animals suggest a seemingly opposite hypothesis: DRN neurons encode expectation and the receipt of rewards (Nakamura et al., 2008; Ranade and Mainen, 2009; Bromberg-Martin et al., 2010; Miyazaki et al., 2011; Inaba et al., 2013). This notion is supported by the facts that electrical (Miliaressis et al., 1975; Miliaressis, 1977) or optogenetic (Liu et al., 2014) stimulation of the DRN induced reward-oriented behavior. How can serotonergic neurons be involved in such opposing processes? One possibility is that different populations of DRN neurons compute appetitive and aversive information separately. Alternatively, single DRN neurons are capable of encoding both appetitive and aversive information.

Using a saccade task, we previously showed that DRN neurons encoded expected reward value continuously throughout given trials (Bromberg-Martin et al., 2010). However, what was lacking in the previous study was an examination of whether the same DRN neurons also respond to aversive stimuli. To answer this question, we made recordings from neurons in the DRN while monkeys were conditioned in a Pavlovian procedure. We used a discrete task with two contexts, one in which rewards may occur (appetitive blocks) and another in which airpuffs may occur (aversive blocks). Additionally, to investigate how DRN neurons encode the expectation of appetitive and aversive stimuli, rewards or airpuffs were probabilistically delivered in each block. These distinct “blocks” enabled us to examine continuous encoding of the emotional contexts as well as value coding of task events with different probabilities. The characteristics of this task were such that, for single DRN neurons, we could directly compare appetitive and aversive information processing throughout the block over different time scales and discrete events.

Materials and Methods

We used both hemispheres of two cynomolgus monkeys (Macaca fascicularis; laboratory designations: monkey H, female, 5.4 kg; monkey S, male, 7.0 kg). All experimental procedures were performed in accordance with the National Institutes of Health Guidelines for the Care and Use of Laboratory Animals and were approved by the institutional Animal Care and Use Committee at Kansai Medical University.

Each monkey was implanted with a post to allow for the maintenance of a stable head position. Eye position was monitored using an infrared video-tracking system at a time resolution of 240 Hz and spatial resolution of 0.1° (EYE-TRAC 6; ASL). Stimuli generated by an active matrix liquid crystal display projector (ELP-505; Epson) were rear projected onto a frontoparallel screen located 60 cm from the monkey's eyes. All aspects of the behavioral experiment, including stimuli presentation, monitoring of eye movements and neuronal activity, and outcome delivery were under the control of a real-time experimentation data acquisition system (Tempo; Reflective Computing).

The activity of single neurons was recorded using tungsten electrodes (diameter, 0.25 mm; impedance, 1–3 MΩ; Frederick Haer). The signal was amplified with a bandpass filter (150 Hz–3 kHz; MEG-5100; Nihon Kohden) and collected at 1 kHz. Single-neuron activity was isolated by a template-matching spike discriminator (time resolution, 50 kHz for waveform matching and spike sampling; Alpha Omega). We also collected spike waveforms for each recorded neuron.

Behavioral task.

The monkeys were trained to learn stimulus–outcome associations in a Pavlovian trace-conditioning task that consisted of the following two distinct blocks of trials: appetitive blocks (Fig. 1A, left) and aversive blocks (Fig. 1A, right). In the appetitive block, three different conditioned stimuli (CSs; green triangle, red open square, and yellow ring for monkey H; red circle, green cross, and blue square for monkey S) were associated with a liquid reward as an unconditioned stimulus (US) with 100%, 50%, and 0% probability, respectively. In the aversive block, three CSs (red circle, blue square, and yellow ring for monkey H; yellow ring, green triangle, and blue square for monkey S) were associated with an airpuff directed at the monkey's face as an aversive US with 100%, 50%, and 0% probability, respectively. The CSs for the “0% reward” and “0% airpuff” trials were identical for both types of blocks. When a reward or an airpuff was not presented (i.e., “0% reward” trials or non-reward trials of the “50% reward” trials in the appetitive block, and “0% airpuff” trials or non-airpuff trials of the “50% airpuff” trials in the aversive block), a tone was sounded. The CSs were 10° in size. For the US, a liquid reward (0.1 ml) was delivered through a spigot that was positioned in front of the monkey's mouth or an airpuff (0.03–0.05 MPa) was delivered through a narrow tube placed 1–2 cm from the face. A tone was presented from a speaker located under the screen. All of these outcomes were delivered for 100 ms.

Figure 1.

Behavioral task and location of the DRN. A, Pavlovian trace-conditioning task with two different blocks. B, Experimental procedure. Once a neuron was isolated, either an appetitive or aversive block (in this example, appetitive) started. After the block was completed, there was a delay of several minutes, and the next block (in this example, aversive) started. The baseline activity of the neuron was defined as the mean firing rate during the 500 ms before the initiation of the appetitive and aversive blocks. C, Average normalized frequencies of behavioral responses in monkey H. The orange lines indicate licking, and the green lines indicate blinking. Shaded areas indicate the SEM. D, Average normalized magnitudes of anticipatory licking and blinking during the late trace period (500 ms before US presentation) in monkey H (top) and monkey S (bottom). Asterisks indicate a significant difference between two points (*p < 0.05, **p < 0.01 after Bonferroni correction). Error bars indicate the SEM. E, Location of the recording chambers targeting the DRN drawn on parasagittal MR images for both monkeys. F, Histological reconstruction of the recording sites on Nissl-stained sections (monkey S). The thick arrows indicate electrode penetration. The small arrows indicate microlesions made after recording from a neuron. The two sections are 450 μm apart. The border of the DRN was drawn based on the study by Molliver (1987). MLF, Medial longitudinal fasciculus; PAG, periaqueductal gray; IV, trochlear nucleus.

Each trial started after the presentation of a timing cue (TC; white circle; size, 1–5°) in the middle of the screen for both blocks. The monkeys were not required to fixate on it. After 1 s, the TC disappeared, and one of the three CSs was presented for 1.5 s. The CS disappeared, and after a 1.0 s delay (trace period) the US was delivered. To examine the responses to unpredicted outcomes, in addition to the cued trials, noncued trials were also included, in which a reward alone (free reward) or a tone alone (free tone) was delivered in the appetitive block, or an airpuff alone (free airpuff) or a tone alone (free tone) was delivered in the aversive block. All trials were presented with a random intertrial interval (ITI) of 7–9 s. One block typically consisted of 64 trials with fixed proportions of trial types (100%, 16 trials; 50%, 16 trials; 0%, 16 trials; free reward or airpuff, 8 trials; free tone; 8 trials).

For each neuron, we collected data by repeating the appetitive and aversive blocks two or more times. The order of the blocks was counterbalanced across the experiments. There was a several minute delay between the blocks, and there was no cue for when and which block would be started (Fig. 1B).

Analyses of behavioral data.

We analyzed anticipatory licking and blinking as behavioral measures to estimate the monkey's valuation of appetitive and aversive stimuli. Licking behavior was detected by a vibration sensor, and the signal was digitized by a threshold-crossing algorithm. Blinking was detected by the saturation of the eye position signal in either the horizontal or vertical axis. The data were normalized as (x − Min/(Max − Min), where x was the mean frequency of licking and blinking for the given task period, and Max and Min were the maximum and minimum frequencies of licking or blinking in the entire recording session, respectively. Because anticipatory licking and blinking were observed most often during the second half of the trace period (Fig. 1C), quantitative comparisons between conditions were made by analyzing the rates of licking and blinking during 500 ms before the US onset. To examine the difference in the frequency of these measures depending on the CS, we performed one-way repeated-measures ANOVA followed by multiple comparisons using Bonferroni correction (Fig. 1D).

Localization of the DRN.

We used the same technique to localize the DRN as has been described previously (Nakamura et al., 2008). A circular CILUX recording chamber (Crist Instruments), angled 35° posteriorly, was implanted over the midline of the parietal cortex to access the brainstem between the superior and inferior colliculi. The location of the DRN was verified by overlaying penetration record maps on magnetic resonance (MR) images (0.3 T; AIRIS; Hitachi). As shown in Figure 1E, MR images were obtained with the recording chamber filled with povidone-iodine ointment, which clarified the relative location of the recording chamber and the underlying brain structure.

During the initial survey of the DRN, the following brain structures were identified and were particularly useful as landmarks: the superior colliculus, with receptive fields in the upper visual field with large eccentricities; the mesencephalic trigeminal nucleus, with responses to mouth movements; and, most importantly, the trochlear nucleus with increased firing during downward eye movements. The DRN is located ventral to the central aqueduct (Molliver, 1987). It has also been reported that, in the primate, the lateral components (wings) of the DRN form the larger division, and the wings are best developed at the level of the trochlear nucleus (Jacobs and Azmitia, 1992). We therefore analyzed neurons that were located (1) 0–2 mm anterior to the trochlear nucleus and (2) 1–2 mm from the midline, all of which were well estimated by neuronal landmarks, as described above.

At the end of the recording session, microlesions were placed in the DRN to histologically identify the recording sites where many task-related neurons were recorded. After completion of all experiments, monkey S was deeply anesthetized with sodium pentobarbital and intracardially perfused with saline followed by 4% paraformaldehyde. The brain was removed and cut in coronal sections (50 μm) for Nissl staining with cresyl violet (Fig. 1F).

Analyses of neuronal activity.

The DRN includes serotonergic and nonserotonergic neurons. Serotonergic neurons reportedly do not always differ significantly from non-serotonergic neurons in terms of their electrophysiological features, such as the baseline firing rate, spike waveform, and irregularity of firing pattern (Allers and Sharp, 2003; Kocsis et al., 2006). Therefore, rather than choosing neurons with specific electrophysiological properties, we unselectively recorded the activity of neurons encountered in the DRN during the experimental sessions. The data were chosen for further analyses off-line, as described below.

For each neuron, we computed the mean firing rates for the following task periods: the ITI during 500 ms before TC onset in the second and later trials; the TC period 0–1000 ms after TC onset in the second and later trials; and the trace period 1000 ms before US onset.

The time window for analyzing CS and US responses was chosen based on their latencies. To measure neural response latencies, we referred to methods reported previously (Maunsell and Gibson, 1992; Morrison and Salzman, 2009) that compared activity following CS or US onset with the preceding TC (for CS) or trace (for US) activity. Specifically, we first constructed a histogram of the number of spikes in 20 ms bins, shifted by 2 ms. Next, we determined which of the 20 ms bins, shifted in 2 ms steps, met a criterion response during the 500 ms after CS or US onset. The criterion for an excitatory (or inhibitory) response was met if at least 10 consecutive overlapping bins contained more (or fewer) spikes than 95% of the activity during the preceding period. Latency was defined as the beginning of the first of these 10 consecutive significantly different bins (20 ms duration). We also performed similar analyses using 20 consecutive bins, which gave consistent latencies.

As 82.5% of the analyzed neurons (203 of 246 neurons) started to change their activity immediately following the CS presentation, the standard window of 100–500 ms after CS onset was applied to these neurons. For 43 of 246 neurons whose response to the CS started later than 400 ms, the activity during the 500–1500 ms after CS onset was analyzed. For each neuron, the same window was used for the appetitive and aversive blocks, blind to the valence. The standard time window for the US responses was 100–500 ms after US onset. For the neurons whose US response latency was very short, 25–125 ms (latency <100 ms) or 50–250 ms (latency <125 ms) windows were used. This was often the case for the response to an airpuff in the aversive block (102 of 246 neurons), but also occurred after rewards in the appetitive block (39 of 246 neurons). Thus, for analyses of the US responses in each block, different windows may have been applied to some neurons, depending on their response latencies (Matsumoto and Hikosaka, 2009a,b).

The magnitudes of the CS and US responses were computed as the mean firing rate during each time window relative to the activity during the preceding period (i.e., for the CS response, the TC period; for the US response, the trace period). However, because DRN neurons often showed significant modulations in TC and trace period activities, we also measured the CS response relative to the ITI, and the US response relative to the ITI and TC activities. The results were essentially consistent regardless of the way that the CS and US responses were measured.

Each neuron was judged to be task related if its activity during at least one of the task periods (ITI, TC, CS, trace, and US) for at least one of the trial conditions (four trial types in the appetitive and aversive blocks) was significantly modulated. We used the Mann–Whitney U test to compare activity between two conditions or time windows, and the Kruskal–Wallis test to compare activity among multiple conditions or time windows. After the Kruskal–Wallis test, if a significant difference was found (p < 0.05), we used the Mann–Whitney U test with Bonferroni correction. We defined activity during a particular task epoch as significantly modulated if (1) it was different from the preceding period or from other trial types (p < 0.01), (2) the outcome probability/predictability produced a significant effect (p < 0.05), or (3) a block-dependent effect was observed (ITI and TC, p < 0.05).

For the task-related neurons, we also examined the graded modulation of CS, trace, and US responses depending on the outcome probability. We first performed Spearman's rank correlation test (p < 0.05). Then, we performed a one-way repeated-measures ANOVA followed by multiple comparisons using Bonferroni correction.

We further computed the “baseline” activity of each neuron, which was defined as the mean firing rate during the 500 ms before TC onset on the first trial of the appetitive and aversive blocks (i.e., before the block was initiated). Because there was no cue for when and which block would be started next, the baseline while the task was stopped between the blocks was regarded as activity “outside of task” in a neutral context. To confirm this hypothesis, we compared the baseline activity before each block using the Wilcoxon signed-rank test. If neurons exhibited temporary bursting activity 500 ms before the initiation of the first trial of the block, we analyzed the mean firing rate during the 1000–500 ms before TC onset to compute the baseline activity.

To generate population histograms, neuronal activity was normalized by computing receiver operating characteristic (ROC) values based on the comparison of the firing rates between the baseline activity and a test window (100 ms duration). The analysis was repeated by moving the test window in 20 ms steps. The dataset for the baseline activity was collected as the mean firing rates of 100 ms windows shifted in 20 ms steps during the 500 ms before the initiation of both appetitive and aversive blocks (i.e., 20 windows for each block). Because the results indicated consistency between the subjects (χ² test, p > 0.05), the data were collapsed across the two monkeys for further analyses.

Analyses of the electrophysiological characteristics of neurons.

We characterized the physiological properties of the sampled neurons, including spike waveform, baseline firing rate, and irregularity of firing pattern. Consistent with previous studies, the typical spike shapes in the current experiment were characterized as follows: first, sharp and small negative; second, sharp and large positive; third, long-duration negative, and fourth, long-duration positive waves (Kocsis et al., 2006; Nakamura et al., 2008). Due to the limited duration of data collection of the spike waveform (up to 2 ms), we measured spike duration from the first sharp negative to the peak of the third, long-duration negative deflection. Nevertheless, among 214 of 246 neurons in which a spike shape was obtained, 114 showed spike durations longer than 2 ms; therefore, we computed the ratio of neurons with spike durations longer than 2 ms. To quantify the irregularity of spike trains, we used an irregularity metric introduced by Davies et al. (2006), which was called “IR” and computed as log (ISI_i/ISI_i+1): the difference between two adjacent interspike intervals (ISIs). A small IR indicates regular firing, and a large IR indicates irregular firing. We then computed the median of all IR values during the whole task period for all trial types.

Results

Behavioral data

As a prelude to determining whether serotonergic neurons are involved in processing appetitive and aversive information, we first showed that the monkeys could discriminate different CSs by comparing behavioral measures: licking a spout in anticipation of a reward and blinking in anticipation of an airpuff during the late trace period, where they were detected most often (Fig. 1C). We also observed that frequent blinking occurred during the entire task periods, including the ITI and TC in the aversive blocks, but not in the appetitive blocks.

The licking frequencies of both monkeys were different among reward CSs in the appetitive block (ANOVA, F_(2,288) = 47.6, p < 0.0001, for monkey H; F_(2,34) = 49.0, p < 0.0001 for monkey S); the blinking frequencies were different among airpuff CSs in the aversive block (ANOVA, F_(2,288) = 19.2, p < 0.0001, for monkey H; F_(2,22) = 9.5, p = 0.001 for monkey S). As shown in Figure 1D, both monkeys licked more often when the reward was more predicted and blinked more often when the airpuff was more predicted. Multiple comparisons revealed significant graded differences in the licking frequencies, depending on reward probability for monkey H (p < 0.0001 for 0% vs 50%; p = 0.007 for 50% vs 100%), but not for monkey S; and the blinking frequencies among airpuff CSs for monkey S (p = 0.01 for 0% vs 50%, p = 0.0049 for 50% vs 100%), but not for monkey H, suggesting individual differences and block dependency in discriminating between the CSs based on outcome probabilities. Compared with the 50% reward CS, there was a significant increase in blinking following the 0% reward CS in both monkeys (p < 0.01 after Bonferroni correction), presumably because of stimulus generalization; blinks were slightly evoked even by the 0% airpuff CS in the aversive block, and the same stimulus was also used as the 0% reward CS in the appetitive block.

Neuronal data

We recorded single-unit activity from a total of 300 neurons (160 from monkey H and 140 from monkey S) in the DRN of both hemispheres of the two monkeys. Among them, we performed analyses for a total of 246 neurons (145 from monkey H and 101 from monkey S) that met the criteria of task-related modulation. To obtain a more detailed picture of neuronal responses so as to determine whether and how single DRN neurons respond to both appetitive and aversive stimuli, we conducted extensive analyses of different aspects of their responses during the Pavlovian conditioning task.

Tonic activity during ITI and TC periods

We first asked whether DRN neurons encoded differential emotional contexts. In the present task, the appetitive and aversive trials were run as a block. Therefore, although the visual stimulus, a white circle, was identical during the TC period in both types of blocks, expected outcomes were different, which may result in differential TC activity depending on the kind of block. We indeed found that >50% of DRN neurons showed discrete activity during the TC period in the appetitive and aversive blocks (Mann–Whitney U test, p < 0.05; Table 1; Fig. 3A).

Table 1.

Context-dependent modulation in TC and ITI activities

TC	ITI	Number of cells
ap > av (n = 109)	ap > av	72
	av > ap	5
	n.s.	32
av > ap (n = 40)	ap > av	1
	av > ap	22
	n.s.	17
n.s. (n = 97)	ap > av	19
	av > ap	7
	n.s.	71

Data are the numbers of neurons that showed a significant difference (p < 0.05) in TC and ITI activities depending on the block. ap, Appetitive block; av, aversive block; n.s., no significant difference.

Figure 3.

Context-dependent modulation during the ITI and TC periods. A, The distribution histogram of the ROC areas comparing TC activity in the appetitive and aversive blocks. The area under the ROC of >0.5 indicates that TC activity was stronger in the appetitive block than in the aversive block. Neurons were sorted into the following three categories based on significant block discrimination during the TC period (p < 0.05): appetitive block preference (blue); aversive block preference (red); and no preference (white). B, The differences in activity between the appetitive and aversive blocks during the ITI and TC periods were positively correlated. C–E, The differences in ITI and TC activities were consistent regardless of the preceding outcomes (i.e., after rewards, tones, or airpuffs). Average normalized ITI and TC activities of neurons with aversive block preference (C), no preference (D), and appetitive block preference (E) during the TC period are shown separately for the appetitive and aversive blocks. The data are further separated depending on the outcomes in the preceding trials. F–H, Proportions of neurons exhibiting significant modulations in ITI and TC activities depending on previous outcomes in the appetitive and aversive blocks.

For example, the neuron shown in Figure 2A exhibited a phasic excitation after TC onset, followed by a tonic elevation of activity in the appetitive block. In the subsequent aversive block, the activity of the same neuron remained low (Fig. 2A, bottom row). Thus, TC activity was significantly higher in the appetitive block than in the aversive block (Mann–Whitney U test, p = 0.1 × 10⁻⁸). There were also neurons with opposite responses. The example neuron in Figure 2B exhibited significantly higher tonic activity during the TC period in the aversive block than in the appetitive block (p = 0.59 × 10⁻¹⁰). Such TC activity of these two neurons was reproduced when the blocks were repeated (data not shown).

Figure 2.

Activity of DRN neurons in the appetitive and aversive blocks. Action potentials are shown by raster plots in the chronological order of each trial type. The changes in firing rate are shown by the perievent histograms smoothed with a Gaussian kernel (δ = 5 ms, width = 5 δ). For each trial type, the data are aligned to CS onset or to outcome onset. The activities on the reward, tone, and airpuff trials are shown in blue, purple, and red, respectively. In the raster, tone trials are shaded in purple. A, This neuron showed stronger tonic activity during the ITI and TC periods of the appetitive block than of the aversive block. It also exhibited a stronger response to the 100% reward CS than to the 50% and 0% reward CSs. B, This neuron showed stronger tonic activity during the ITI and TC periods of the aversive block than of the appetitive block. It also exhibited a stronger inhibition to the 100% reward CS than to the 50% and 0% reward CSs. C, This neuron showed modulation in the CS response by outcome probability in both the appetitive and aversive blocks.

The differences in TC activity between the blocks may reflect differences in the appetitive and aversive contexts. If that were the case, the activity during the ITI may also be modulated by contexts. Indeed, this block-dependent modulation was observed in ITI activity. Among 149 neurons that showed a significant difference in block-dependent TC activity, 94 showed a consistent and significant difference in ITI activity (Mann–Whitney U test, p < 0.05; Table 1). Moreover, there was a positive correlation in block discrimination between ITI and TC activities (Spearman's rank correlation, ρ = 0.73, p = 0.12 × 10⁻⁴²; Fig. 3B), indicating that if a neuron showed stronger ITI activity in the appetitive block than in the aversive block, it also showed stronger TC activity in the appetitive block than in the aversive block, and vice versa.

While the block-dependent differences in TC activity suggest that DRN neurons discriminate emotional contexts, the results might also be explained by behavioral distinctions between the blocks because the monkeys' blinking and licking behaviors were discrepant depending on the block during the ITI and TC periods (Fig. 1C), and motor aspects of the raphe nuclei have been reported repeatedly (Jacobs and Fornal, 1993; Ranade and Mainen, 2009; Li et al., 2013).

To examine this issue, we first analyzed, for each neuron, whether there was a significant correlation between licking frequency and neuronal activity during the TC period for each block. If licking behavior were the cause of differences in TC activity, neurons with appetitive (aversive) block preference in TC activity should show a positive (negative) correlation between TC activity and licking frequency. Statistical analyses revealed that a small number of neurons had a significant correlation between licking frequency and TC activity (Spearman's rank correlation, p < 0.05). Among 54 appetitive block-preferring neurons recorded when a sufficient licking signal was measured, 7 showed a positive correlation and 1 showed a negative correlation between licking frequency and TC activity in the appetitive block. Among 27 aversive block-preferring neurons, 3 showed a negative correlation between licking frequency and TC activity. However, for most other neurons, including the one in Figure 2A, TC activity was not significantly correlated with licking frequency in the appetitive block (ρ = −0.009, p = 0.95), indicating that licking had only a minimal effect on the block-dependent modulation in TC activity.

We next analyzed whether different eye positions or blinking behaviors, as shown in Figure 1C, were the cause of the block-dependent differences by comparing TC activity between the appetitive and aversive blocks, but now separately in trials in which the monkeys' eye positions were within or outside (including blinking) the central 16° window. All 120 neurons, with a sufficient number of trials in which blinking occurred, exhibited consistent differences in TC activity between the blocks regardless of the position of the eyes (Mann–Whitney U test, p < 0.05), suggesting that eye position and blinking were not the cause of differential TC activity.

The block-dependent differences in TC and ITI activities might also be due to the residual effects of outcomes in the previous trials, as many DRN neurons respond tonically to outcomes (Nakamura et al., 2008). To control for this possibility, we broke down the data into two groups depending on the preceding outcomes: rewards and tones in the appetitive block; and airpuffs and tones in the aversive block. As shown in Figure 3C–E, the preceding outcomes had only a modest effect on TC and ITI activities. At the single-neuron level, some neurons showed significant differences in their activities depending on outcomes in the previous trials (Mann–Whitney U test, p < 0.05; Fig. 3F–H). The aversive-preferring neurons were likely to fire more strongly after relatively negative outcomes (i.e., tones in the appetitive block and airpuffs in the aversive block) than after positive ones (i.e., rewards) during the ITI and TC periods (Fig. 3F), while the appetitive-preferring neurons were likely to fire more strongly after positive outcomes than after negative ones in the appetitive block (Fig. 3H). Nevertheless, the analysis revealed that the differences in ITI and TC activities were predominantly consistent regardless of the preceding outcome in each preference type (Fig. 3C–E). Together, these results suggest that DRN neurons exclusively encoded appetitive and aversive contexts during the ITI and TC periods, rather than behavioral differences or information of the preceding outcome.

Changes in ITI and TC activities relative to baseline

We have revealed that DRN neurons discriminated the appetitive and aversive blocks even during the ITI. This predicts that ITI activity should change, on a trial-by-trial basis, when compared with the baseline measured outside of the task. In our task, there was no clue as to what the upcoming block was before each block was initiated. Thus, the monkeys should not discriminate between the appetitive and aversive blocks during the “baseline epoch” (500 ms before the first trial started). In support of this idea, a comparison of the baseline among the blocks revealed no significant differences across neurons, suggesting that the baseline activity was not yet modulated depending on the blocks (Wilcoxon signed-rank test, p = 0.1). By comparing the activity 1000–500 ms before TC onset and 500 ms before TC onset on the first trial, we also confirmed that monkeys did not predict the timing of the initiation of a block (Wilcoxon signed-rank test: p = 0.8 before the appetitive, p = 0.2 before the aversive block started).

To examine how the activity was modulated during the ITI, we analyzed trial-by-trial changes in ITI activity relative to the baseline (Fig. 4). As the trials progressed in the aversive block, neurons that preferred the aversive block showed a gradual increase in ITI activity that remained positive relative to the baseline. In the appetitive block, the same neurons initially exhibited increased activity, which soon decreased to around the baseline level (Fig. 4A). Conversely, DRN neurons that preferred the appetitive block showed a gradual decrease in ITI activity in the aversive block, while the same neurons exhibited increased activity in the appetitive block, followed by a decrease in activity during the later trials of the blocks (Fig. 4B). An example neuron with appetitive block preference is shown in Figure 4C. During the earlier trials, ITI and TC activities gradually increased in the appetitive block and decreased in the aversive block. In summary, many DRN neurons discriminated the appetitive and aversive blocks by tonic modulation of the activity during the ITI and TC periods. Such context-dependent tonic modulation in activity occurred, on trial-by-trial bases, after the particular block started.

Figure 4.

Trial-by-trial changes in context-dependent modulation of ITI activity. A, B, Averaged (±SEM) ITI activity relative to the baseline before the initiation of the block (therefore, the data of the first trial were zero) for neurons with aversive block preference (A) and appetitive block preference (B) in TC activity is shown separately for the appetitive (blue) and aversive (red) blocks. Data were smoothed by 3-point averaging except on the first trial. C, An example neuron from B. The raster plots show ITI, TC, and CS activities during the first 15 trials including all trial types in the appetitive (blue) and aversive (red) blocks.

Responses to CSs

DRN neurons responded to the CSs. Among 246 task-related neurons, 114 (46.3%) showed significant responses to the CSs in the appetitive block only, 35 (14.2%) to the CSs in both the appetitive and aversive blocks, and 10 (4.1%) to the CSs in the aversive block only (Mann–Whitney U test, p < 0.05 after Bonferroni correction). Thus, a significant response to the CSs was primarily observed in the appetitive block rather than in the aversive block.

The CS responses of many neurons were modulated by the outcome probability. For example, relative to its activity during the TC period, the neuron in Figure 2A exhibited a significant excitatory response to the 100% CS (Mann–Whitney U test, p = 0.1 × 10⁻⁴) and an inhibitory response to the 0% CS (p = 0.9 × 10⁻³), but not to the 50% CS. Thus, there was a significant positive correlation between reward probability and CS responses (Spearman's rank correlation, ρ = 0.84, p = 0.1 × 10¹²). The neuron in Figure 2B also responded to the reward CSs, but in a mirror image manner; when compared with its response during the TC period, it showed inhibition in the response to the 100% reward CS (p = 0.0002) and strong excitation to the 0% reward CS (p = 0.1 × 10⁻⁵), resulting in a significant negative correlation between reward probability and CS response (ρ = −0.87, p = 0.5 × 10⁻¹⁷). Neither neuron, however, exhibited significant responses to any of the CSs associated with an airpuff. A small number of neurons, like the one in Figure 2C, showed significant modulation in the CS response, depending on airpuff probability (ρ = −0.47, p = 0.001).

In the appetitive block, 60 neurons (24.4%) and 55 neurons (22.4%) showed significant positive (100% > 50% > 0% CSs; Fig. 5A) and negative (100% < 50% < 0% CSs; Fig. 5C) correlations between reward probability and CS response, respectively (Spearman's rank correlation, p < 0.05). Compared with TC activity, the CS responses of these neurons were strongly inhibited by the 0% reward CS and excited by the 50% and 100% reward CSs (Fig. 5B), and vice versa (Fig. 5D). As a population, both types of neurons exhibited significantly different responses to reward CSs (ANOVA: F_(2,118) = 41.4, p < 0.0001 for positive correlation; F_(2,108) = 42.2, p < 0.0001 for negative correlation). Multiple comparisons revealed significant differences in the CS responses, depending on reward probability for neurons with positive correlations (p < 0.0001 for 0% vs 50%; p = 0.0023 for 50% vs 100%; Fig. 5B) and negative correlations (p < 0.0001 for 0% vs 50%; p = 0.01 for 50% vs 100%; Fig. 5D).

Figure 5.

Responses to CSs. A, Average normalized activity of 60 neurons that exhibited positive correlations between reward probability and CS response (p < 0.05). Histograms are shown for 100% (blue), 50% (cyan), and 0% (gray) reward CSs. B, The magnitudes of the mean (±SEM) responses of the positive reward CS coding neurons. Asterisks indicate significant differences between responses to two CSs (*p < 0.05, **p < 0.01 after Bonferroni correction). C, D, Same as A and B, but of 55 negative reward CS coding neurons. E, F, Averaged activity of 12 positive airpuff CS coding neurons. Histograms are shown for 100% (red), 50% (pink), and 0% (gray) airpuff CSs. G, H, Same as E and F, but of 12 negative airpuff CS coding neurons. I, Correlation coefficients between the outcome probability (i.e., 0%, 50%, and 100%) and CS responses. The abscissa indicates the correlation coefficient between reward probability and CS response. The ordinate indicates the correlation coefficient between airpuff probability and CS response. The open circles, squares, and black dots indicate neuronal responses with a statistically significant correlation with reward probability only, airpuff probability only, and both, respectively (p < 0.05); gray dots are neurons with no significant correlations. The marginal histograms show the distributions of the correlation coefficients. The white and black bars indicate neurons with a statistically significant correlation for reward or airpuff probability only and both, respectively. The gray bars indicate no significant correlation.

In the aversive block, 12 neurons (4.9%) and 12 neurons (4.9%) showed significant positive correlations (100% > 50% > 0% CSs; Fig. 5E) and negative correlations (100% < 50% < 0% CSs; Fig. 5G) between airpuff probability and CS response, respectively (Spearman's rank correlation, p < 0.05). As a population, these neurons exhibited a significant difference in the CS responses relative to TC activity (ANOVA: F_(2,22) = 33.2, p < 0.0001 for positive correlation; F_(2,22) = 23.9, p < 0.0001 for negative correlation). Multiple comparisons revealed significantly different responses to the aversive CSs depending on airpuff probability for neurons with positive correlations (p = 0.0002 for 0% vs 50%; p = 0.0017 for 50% vs 100%; Fig. 5F) and negative correlations (p = 0.0046 for 0% vs 50%; p = 0.0011 for 50% vs 100%; Fig. 5H). These results suggest that DRN neurons are involved in graded coding of values associated with reward or airpuff CSs. However, such value coding was more prominent in the appetitive than in the aversive blocks.

We further asked whether single DRN neurons similarly encoded the values of both appetitive and aversive CSs. We found this to be only partially true. First, there was a significant positive correlation between the response to the 0% reward (“worst”) CS in the appetitive block and the response to the 100% airpuff (“worst”) CS in the aversive block (Spearman's rank correlation, ρ = 0.3, p = 0.7 × 10⁻⁶). However, the correlation between the response to the 100% reward (“best”) CS in the appetitive block and the response to the 0% airpuff (“best”) CS in the aversive block was not significant. Finally, whereas the CS responses of many DRN neurons were modulated by reward probability in the appetitive block, the CS responses of the same neurons did not encode airpuff probability on a similar value scale in the aversive block. In the joint distribution of correlation coefficients for the appetitive and aversive blocks (Fig. 5I), the top-right and bottom-left quadrants exhibit a straightforward correlation between CS value coding in the appetitive and aversive blocks. For example, if reward and airpuff CSs are evaluated on a common value scale in the DRN, then a neuron with a stronger response to the 100% reward (“better”) CS than to the 0% reward (“worse”) CS in the appetitive block (positive correlation coefficient between reward probability and neuronal firing rate; Fig. 5I, right half) should respond more strongly to the 0% airpuff (“better”) CS than to the 100% airpuff (“worse”) CS in the aversive block (Fig. 5I, top half). Instead, there was a small, but significant, correlation for neurons that had stronger responses to the 100% reward CS that also showed stronger responses to the 100% airpuff CS; similarly, neurons that had stronger responses to the 0% reward CS also showed stronger responses to the 0% airpuff CS (Spearman's rank correlation, ρ = 0.15, p = 0.02).

The TC and CS responses were often jointly modulated (Table 2). Neurons with appetitive block preference in TC activity tended to exhibit stronger responses to the 100% reward CS than to the 50% and 0% reward CSs. Conversely, neurons with aversive block preference during the TC period tended to exhibit stronger CS responses to the 0% reward CS than to the 50% and 100% reward CSs.

Table 2.

Modulation of neuronal activity by CS-US probability

TC type	Appetitive CS		Appetitive US		Aversive CS		Aversive US		Reward omission
	100% > 0%	0% > 100%	Free >100%	100% > Free	100% > 0%	0% > 100%	Free >100%	100% > Free	Increase	Decrease
ap > av	39	15	39	16	7	3	10	7	9	27
(n = 109)	(35.8)	(13.8)	(35.8)	(14.7)	(6.4)	(2.8)	(9.1)	(6.4)	(8.3)	(24.8)
av > ap	2	14	9	3	3	1	2	3	8	1
(n = 40)	(5.0)	(35.0)	(22.5)	(7.5)	(7.5)	(2.5)	(5.0)	(7.5)	(20.0)	(2.5)
n.s.	19	26	19	9	2	8	10	6	14	6
(n = 97)	(19.6)	(26.8)	(19.6)	(9.3)	(2.1)	(8.2)	(10.3)	(6.2)	(14.4)	(6.2)
Total	60	55	67	28	12	12	22	16	31	34

Number (%) of neurons that showed a significant modulation in the CS response depending on the reward/airpuff probability, in the US response depending on the reward/airpuff predictability, and in the response to reward omission. The data are shown for separate groups of neurons classified by preferred activity during the TC period. ap, Appetitive block; av, aversive block; n.s., no significant difference.

This was confirmed statistically at the single-neuron level (Fig. 6). We used ROC analysis to measure the change in firing rate of each neuron during the TC period. The ROC area was 0 if the neuron was strongly inhibited, 0.5 if it maintained its ITI firing rate, and 1 if it was strongly excited. We also examined the correlation coefficients between US probability and CS response. This coefficient was positive if the CS response was stronger when US probability was higher; it was negative if the CS response was lower when US probability was higher. We found that the activity of a neuron during the TC period was positively correlated with its degree of CS discrimination in the appetitive block (Spearman's rank correlation, p < 0.01; Fig. 6A). In the aversive context, however, changes in TC activity relative to the ITI as well as changes in the CS responses were both small and did not show any significant correlation with one another (Fig. 6B).

Figure 6.

Correlated modulation of TC activity and CS responses. A, Significant correlation between CS discrimination and changes in TC activity relative to ITI activity in the appetitive block. The blue and red dots and open circles indicate neurons with appetitive and aversive block preferences or no preferences in TC activity, respectively. B, Same as A, but in the aversive block. There were no significant correlations between CS discrimination and changes in TC activity.

To summarize, CS value coding by DRN neurons was much stronger in the appetitive than in the aversive blocks. The strength of CS value coding was positively correlated with changes in activity during the TC period in the appetitive blocks, but not in the aversive blocks.

Responses to USs

Many neurons also responded to the USs (Fig. 7A). Unlike their responses to the CSs, the US responses were observed in both blocks: 74 neurons (30.1%), 43 neurons (17.5%), and 88 neurons (35.8%), respectively, showed significant responses to rewards only, airpuffs only, and both rewards and airpuffs. We first asked whether the responses of DRN neurons to the USs encoded their value on a single dimension. If that were the case, the responses to rewards and airpuffs should be in the opposite direction (e.g., excitatory for rewards and inhibitory for airpuffs, compared with the trace period). Instead, many DRN neurons responded to rewards and airpuffs in the same direction, and responses to free rewards and free airpuffs were positively correlated (Spearman's rank correlation, ρ = 0.27, p = 0.26 × 10⁻⁶; Fig. 7B).

Figure 7.

Responses to USs. A, Proportions of neurons that showed significant changes in excitatory (white) or inhibitory (gray) responses to the USs and tones relative to the preceding trace period activity (p < 0.05). The data are shown separately for outcome predictability (i.e., 100%, 50%, and 0%) in the appetitive (left) and aversive (right) blocks. B, The directions of changes in the response to free rewards and airpuffs were the same for many neurons. C, Changes in the US response by the preceding CS. The abscissa indicates the correlation coefficient between the reward predictability implied by appetitive CSs and the reward response. The ordinate indicates the correlation coefficient between the airpuff predictability implied by aversive CSs and the airpuff response. The open circles, squares, and black dots indicate neurons with a statistically significant correlation for the appetitive block only, aversive block only, and both, respectively (p < 0.05); gray dots are neurons with no significant correlations. The marginal histograms show the distributions of the correlation coefficients. The white and black bars indicate neurons with statistically significant correlations for reward or airpuff only and for both, respectively. The gray bars indicate no significant correlation.

The US responses were strongly modulated by the preceding CSs. The example neuron in Figure 2A showed the strongest phasic responses to free, unpredicted rewards (Mann–Whitney U test, p = 0.0003), and a significant correlation between the US response and outcome predictability indicated by the CS (ρ = −0.40, p = 0.02). We also computed the correlation coefficient between them for each neuron, separately for the appetitive and aversive blocks (Fig. 7C). Changes in the US responses depending on the preceding CSs were commonly observed in the appetitive block: 67 neurons (27.2%) responded more strongly to an unpredicted reward than to a predicted reward, whereas 28 neurons (11.4%) responded more strongly to a predicted reward than to an unpredicted reward (Spearman's rank correlation, p < 0.05). As a population, the correlation coefficient between the response to a reward and its predictability was significantly negative (sign test, median correlation coefficient = −0.13, p = 0.34 × 10⁻⁶), indicating a stronger response to an unpredicted reward than to a predicted reward. Such CS-dependent modulation in the US response was less common in the aversive block: 22 neurons (8.9%) responded more strongly to an unpredicted airpuff than to a predicted airpuff, whereas 18 neurons (7.3%) responded more strongly to a predicted airpuff than to an unpredicted airpuff. As a population, there was no significant bias to either a positive or negative correlation coefficient (sign test, median correlation coefficient = −0.04, p = 0.07).

We also found that the predictability-dependent modulations in the responses to rewards were positively correlated with those to airpuffs (Spearman's rank correlation, ρ = 0.22, p = 0.0004). In other words, neurons that exhibited a stronger response to an unpredicted reward than to a predicted reward showed a stronger response to an unpredicted airpuff than to a predicted airpuff (Fig. 7C).

Responses to reward omission

Although a tone was a neutral outcome, it was relatively negative in the appetitive block. On the other hand, the same tone was relatively positive in the aversive block. Hence, we examined in more detail how DRN neurons responded to the tone. As shown in Figure 7A, only a small number of neurons responded to tones in the 0% reward trials (i.e., 100% predictability; n = 15, 6.1%) and free tone trials (i.e., 0% predictability; n = 11, 4.5%) in the appetitive block. However, many neurons (n = 66, 26.8%) exhibited significant excitatory or inhibitory modulation in the response to a tone in the 50% reward trials, when a reward was partly expected following the 50% reward CS, but was not given, and a tone was presented.

To characterize the neurons that responded to reward omission, we analyzed 65 neurons whose response to a tone in the 50% reward trials was (1) significantly modulated relative to the preceding trace period and (2) significantly different from the response to a free tone in the appetitive block (Mann–Whitney U test, p < 0.05). Among these, 31 showed increases (Fig. 8A) and 34 showed decreases (Fig. 8B) in their responses to reward omission or a tone in the 50% reward trials. The modulation of activity in these neurons had several characteristics. First, they did not demonstrate a significant response to the same tone when it was presented as expected (0% reward) or unexpectedly (free tone), or to any tones in the aversive block. Second, these neurons also showed modulation in TC activity and CS responses, with consistent value coding in the response to the reward omission. The neurons that showed excitation to reward omission were tonically inhibited during the TC period and exhibited negative reward coding in their CS responses (i.e., stronger response to the 0% reward CS than to the 100% reward CS; Fig. 8A). Conversely, neurons like the one in Figure 2A that showed inhibition to the tone in the 50% reward trials exhibited enhanced activity during the TC period and positive coding in their CS responses in the appetitive block (i.e., stronger response to the 100% reward CS than to the 0% reward CS; Fig. 8B).

Figure 8.

Responses to reward omission. A, B, Average normalized activity of neurons whose response to the omission of an expected reward (i.e., a tone in the 50% reward trials) was significantly more (A) or less (B) than the immediately preceding trace period activity and significantly different from the response to a free tone in the appetitive block (p < 0.05). The three columns on the left show activity in the appetitive block; those on the right show activity in the aversive block. For each, the data are aligned to CS onset (left), reward or airpuff onset (middle), and tone onset (right).

Appetitive and aversive information coding by different groups of DRN neurons

Table 2 summarizes the different modulations in the CS and US responses depending on the outcome probability. Many neurons with appetitive block preference in TC activity responded more strongly to the 100% reward CS than to the 0% reward CS and tended to be inhibited by reward omission. Conversely, many neurons with aversive block preference in TC activity responded more strongly to the 0% reward CS than the 100% reward CS and tended to be excited by reward omission. These mirror image responses were not observed for the US responses. Such distinct activity profiles by different groups of neurons might be associated with differences in electrophysiological characteristics, including baseline activity, spike width, and irregularity of firing pattern; however, there were no significant differences among the groups (Table 3).

Table 3.

Electrophysiological properties of DRN neurons

TC type	Baseline	Spike width ≥2 ms	Irregularity metric
ap > av	7.0 (12.0) Hz	48/92 (52.2%)	0.65 (0.37)
av > ap	8.0 (10.5) Hz	19/34 (55.9%)	0.82 (0.29)
n.s.	7.0 (9.0) Hz	47/88 (53.4%)	0.72 (0.43)

The physiological properties of the recorded neurons characterized by their baseline firing rate, spike duration, and irregularity of spike trains. For the baseline firing rate and irregularity metric, the median and interquartile range (in parentheses) are shown. For spike duration, only neurons whose spike shape was recorded were analyzed; the number (%) of neurons with duration longer than 2 ms is shown. ap, Appetitive block; av, aversive block; n.s., no significant difference.

Discussion

Using a Pavlovian conditioning paradigm, we obtained evidence that single DRN neurons are involved in computing both appetitive and aversive information. Specifically, DRN neurons discriminated emotional contexts by tonically modulating their activity. In the appetitive block, they further kept track of reward values associated with CSs and USs. In the aversive block, such probability-dependent responses were observed less often. Rather, many neurons continued to encode the aversive context in tonic activity. Thus, the activity of DRN neurons is characterized by dual processes: a tonic and categorical one to discriminate emotional contexts; and a relatively phasic, quantitative one to encode rewarding events.

These dual processes that are dependent on contexts stand in strong contrast to the activity of midbrain dopamine neurons. The dopamine neurons typically show phasic responses to the relative value of appetitive and aversive stimuli; however, sustained modulation such as block-dependent ITI and TC activities was much less common (Schultz, 1998; Matsumoto and Hikosaka, 2009b). Therefore, while midbrain dopaminergic neurons are suitable for updating appetitive and aversive values in stimulus–outcome associations, DRN neurons may be better suited for encoding the overall value of the context.

Context-dependent tonic modulation

Analyses of tonic activity during the ITI and TC periods revealed that more than half of DRN neurons discriminated the appetitive and aversive blocks. Context-dependent activity in the DRN was also demonstrated by comparing reward and non-reward contexts (Li et al., 2013). However, even though DRN neurons were reported to process both appetitive and aversive information, their activity in the aversive context was never compared directly to that in the appetitive context.

The modulations of ITI and TC activities followed different courses, depending on contexts (Fig. 3C,E). In the appetitive block, if a neuron showed significant context dependency in ITI activity, the activity changed further during the TC period and had a strong positive correlation with changes in the subsequent CS response, indicating its participation in encoding valence during the course of a trial. In the aversive block, the same neuron showed tonic modulation starting at the ITI that remained stable throughout a trial, suggesting that tonic activity discriminated context rather than single trials and events. One of the possible functions of a context signal in the DRN may be the control of sensitivity to aversive stimuli. This is supported by evidence that the lowering serotonin levels enhances punishment signals (Cools et al., 2008) or cognitive engagement with aversive stimuli (Dayan and Huys, 2008). Moreover, serotonergic modulation of amygdala function may contribute to such control, as serotonergic projections from the DRN inhibit aversive sensory information processing in the amygdala via immune activation (Lowry et al., 2007) or corticotropin-releasing factor (Lowry et al., 2000; Hammack et al., 2002).

A particularly relevant source of tonic activity in the DRN could be the ventromedial prefrontal cortex (vmPFC). In a similar task, neurons in subregions of the vmPFC showed comparable activities, including tonic modulation during an appetitive-preferring block (ventral vmPFC) or an aversive-preferring block (dorsal vmPFC; Monosov and Hikosaka, 2012). The mechanisms controlling the context-dependent neuronal activity may also involve projections from the PFC and habenula to the DRN, which have often been discussed in relation to stressor controllability. DRN neurons are activated by aversive events (Pezzone et al., 1993; Grahn et al., 1999; Takase et al., 2004; Schweimer and Ungless, 2010). However, once the PFC detects that a stressor is controllable, it inhibits the aversive stimulus-driven DRN activity (Amat et al., 2001). Conversely, if a stressor is uncontrollable, projections from the habenula cause increases in serotonin levels that lead to action inhibition.

A recent study found that optogenetic activation of the PFC–DRN pathway was causally involved in increased effortful movement in rats during forced swimming, a challenging and inescapable situation, whereas activation of the PFC–habenula pathway had the opposite effect (Warden et al., 2012). Although we compared appetitive to aversive Pavlovian responses rather than controllability using the current task, PFC–DRN and habenula–DRN interactions might also be important for the regulation of emotional behavior during conditioning in the aversive context.

Responsiveness to CSs and USs

The responses to aversive CSs were scarce even though many DRN neurons phasically responded to airpuffs. This finding was unexpected because the DRN was implicated in aversive information processing in previous studies. We showed that the CS responses were rarely modulated by airpuff probability in the aversive block. However, the monkeys behaviorally distinguished the aversive CSs. Thus, this distinction may be performed by other brain structures such as the lateral habenula. While monkeys were conditioned in a Pavlovian procedure, habenula neurons discriminated the aversive CSs probabilistically associated with airpuffs (Matsumoto and Hikosaka, 2009a). Additional candidate areas that might play a key role in evaluating aversive stimuli are the orbitofrontal cortex (Hosokawa et al., 2007; Morrison and Salzman, 2009) and the amygdala (Paton et al., 2006). On the other hand, the results are consistent with the finding that serotonin depletion did not affect fear conditioning to an explicit CS in rats (Wilkinson et al., 1995). In humans, lowering serotonin levels reduced punishment-induced inhibition, indicating that the serotonergic system is crucial for linking response inhibition with predictions of aversive outcomes rather than aversive information processing per se (Crockett et al., 2009). Together, our data suggest that DRN neurons are involved in the coding of aversive context, but not of detailed stimulus contingencies.

In contrast, the activity of the same DRN neurons was richly modulated in the appetitive context. First, many DRN neurons showed reward probability-dependent modulation in the CS responses. There was also a strong positive correlation between TC activity and CS value coding, which is consistent with our previous finding that DRN neurons keep track of expected rewards throughout the task (Bromberg-Martin et al., 2010). Second, DRN neurons showed reward probability-dependent modulation in the US responses. Specifically, many responded stronger when rewards were unpredicted than when predicted (Fig. 7C). Some also responded to the omission of expected rewards (Fig. 8). This is partly in agreement with the theoretical prediction of serotonergic function proposed by Daw et al. (2002), which has never been examined experimentally at the single-neuron level. They predicted that a phasic serotonergic signal might report a negative RPE, implying that actual punishments are stronger than predicted punishments, such as the omission of rewards. However, we found that DRN neurons that responded to the omission of rewards did not exhibit (1) excitatory responses to free tones in the appetitive block, (2) excitatory responses to aversive CSs and USs that were modulated by airpuff probability in the aversive block, or (3) inhibitory responses to rewards. Therefore, the responses to the omission of an expected reward observed in some DRN neurons cannot be fully explained by a negative RPE signal hypothesis in the traditional reinforcement–learning framework.

Furthermore, the present findings indicate that value coding in the responses to CSs and USs in the appetitive and aversive blocks was not simply scaled in the opposite direction. Neurons that responded more strongly to a “better” CS than to a “worse” one in the appetitive block (100% > 0% reward) did not respond more strongly to a “better” CS than to a “worse” one in the aversive block (0% > 100% airpuff). Instead, they tended to respond more strongly to the 100% airpuff CS than to the 0% airpuff CS. Likewise, neurons that responded more strongly to unpredicted rewards also responded more strongly to unpredicted airpuffs. Thus, the responses of DRN neurons may be influenced by multiple factors including valence, but also by salience of the stimuli.

Appetitive and aversive information processing in the DRN

We identified at least three distinct types of TC activity modulation: appetitive preferring, aversive preferring, and no preference. Neurons in each group also exhibited different modulations of their CS and US responses (Table 2), indicating functionally distinct populations of cells. The DRN is a heterogeneous structure consisting of neurons with different electrophysiological, neurochemical, and anatomical properties. Several neurotransmitters other than serotonin are also present in the DRN, including GABA, dopamine, glutamate, nitric oxide, substance P, and neuropeptide Y (Charara and Parent, 1998; Michelsen et al., 2007). Different populations of DRN neurons have afferent and efferent connections with distinct brain areas (for review, see Nakamura, 2013). Thus, functional differences may be attributed to neurochemical and/or anatomical characteristics of neuron populations in the DRN. In the present study, we did not determine the chemical identities of recorded neurons. Nevertheless, the activity we measured should still provide important insights into the roles of serotonin in emotional information processing for the following reasons. First, a substantial proportion of DRN neurons are serotonergic: 30% in rats (Descarries et al., 1982); 70% of medium-sized DRN neurons in cats (Wiklund et al., 1981); and 70% in humans (Baker et al., 1991). Second, extensive interconnectivity exists among neurons with different chemical characteristics in the DRN (Weissbourd et al., 2014).

Although previous results concerning the involvement of the serotonergic system in appetitive and/or aversive processing are inconsistent, we found that single DRN neurons encode both appetitive and aversive information using different time scales. This may be due partly to methodological differences. Commonly used pharmacological manipulations generate relatively global, continuous modulation of serotonin levels, which affect the sensitivity to aversive stimuli. Conversely, electrophysiological measures, including single-unit recording or local stimulation, allow us to analyze neural data on a finer time scale. Manipulations focusing on neurotransmitters and anatomical connections, together with analyses of correlations between neuronal activity and behavioral changes, as demonstrated in this study, will reveal more detailed, circuit-level mechanisms of appetitive and aversive information processing by the serotonergic system.

In conclusion, DRN neurons encode emotional contexts and events reflecting different time scales of prediction depending on their valence, as follows: tonic discrimination of the appetitive and aversive blocks; and phasic representation of reward value throughout the trials. Our findings may provide the neural basis of positive and negative emotional information processing required for value-based decision making in differential contexts.