Neurons in the Primate Medial Basal Forebrain Signal Combined Information about Reward Uncertainty, Value, and Punishment Anticipation

Abstract

It has been suggested that the basal forebrain (BF) exerts strong influences on the formation of memory and behavior. However, what information is used for the memory-behavior formation is unclear. We found that a population of neurons in the medial BF (medial septum and diagonal band of Broca) of macaque monkeys encodes a unique combination of information: reward uncertainty, expected reward value, anticipation of punishment, and unexpected reward and punishment. The results were obtained while the monkeys were expecting (often with uncertainty) a rewarding or punishing outcome during a Pavlovian procedure, or unexpectedly received an outcome outside the procedure. In vivo anterograde tracing using manganese-enhanced MRI suggested that the major recipient of these signals is the intermediate hippocampal formation. Based on these findings, we hypothesize that the medial BF identifies various contexts and outcomes that are critical for memory processing in the hippocampal formation.

Introduction

The basal forebrain (BF) is thought to be a key brain area for the control of cognitive functions, such as learning and memory (Mesulam et al., 1983; Everitt et al., 1988; Voytko, 1996; Everitt and Robbins, 1997; Semba, 2000; Zaborszky et al., 2015). Lesions of BF in humans produce deficits in new memory formation and recall (Damasio et al., 1985; Morris et al., 1992; Abe et al., 1998; Solcà et al., 2015), and the severity of mnemonic symptoms in Alzheimer's patients is often related to BF degeneration (Whitehouse et al., 1982; Kesner, 1988).

Animal studies further support the wide role of the BF in the control of computational resources for a range of cognitive functions, such as perception, attention, learning, and memory (Berry and Thompson, 1979; Ridley et al., 1989; Muir et al., 1993; Hasselmo et al., 1996; Voytko, 1996; Everitt and Robbins, 1997; Ridley et al., 1999a, b; Savage et al., 2007; Baxter and Bucci, 2013) and single out the medial regions of the BF (mBF) as particularly important for many kinds of new learning and long-term memory formation. mBF lesions have been found to impede classical aversive learning (Berry and Thompson, 1979) and have also been shown to influence associative memory formation (Hasselmo et al., 1996; Everitt and Robbins, 1997). However, what types of information mBF neurons send to its target structures to modulate learning and memory is unknown.

Learning and memory are known to be disproportionally shaped by surprising events or uncertain contexts (Pearce and Hall, 1980; Yu and Dayan, 2005; Courville et al., 2006; Esber and Haselgrove, 2011; Le Pelley et al., 2011; Bach and Dolan, 2012; Ogawa et al., 2013), as well as by other factors, such as the expectation of a reward or the fear of an aversive stimulus (Eccleston and Crombez, 1999; Bromberg-Martin et al., 2010; McNally et al., 2011; Moore et al., 2012; Anderson, 2013; Wiech and Tracey, 2013). We therefore recorded activity of single mBF neurons while monkeys experienced contexts known to activate and modulate a wide range of learning and memory functions. We found that a population of neurons in the mBF combined information about uncertainty, reward size, punishment, and surprise. We also found that the region of the mBF where these neurons were clustered projects to the hippocampal formation (HF). Based on these findings, we propose a mechanism through which the mBF may control learning and memory functions of the HF.

Materials and Methods

General procedures.

Five adult male rhesus monkeys (Macaca mulatta) were used for the experiments (Monkeys H, P, T, Sm, and S). All procedures for animal care and experimentation were approved by the Animal Care and Use Committee of the National Eye Institute and complied with the Public Health Service Policy on the humane care and use of laboratory animals. A plastic head holder and plastic recording chamber were fixed to the skull under general anesthesia and sterile surgical conditions. The chambers were tilted laterally by 35° and aimed at the ventromedial prefrontal cortex and the anterior portion of the caudate nucleus. Two search coils were surgically placed under the conjunctiva of the eyes. After the monkeys recovered from surgery, they were conditioned using Pavlovian procedures (Experiments 1–5). During the Pavlovian procedures, we recorded the activity of single neurons in the basal forebrain–septum complex.

Neuronal recording.

While the monkey was participating in the task, we recorded the activity of single neurons in the septum and the diagonal band of Broca. The recording sites were determined with 1 mm spacing grid system, with the aid of MR images (4.7T, Bruker) obtained along the direction of the recording chamber. This MRI-based estimation of neuron recording locations was aided by custom software (Daye et al., 2013).

Single-unit recording was performed using epoxy-coated electrodes (FHC). The electrode was inserted into the brain through a stainless-steel guide tube and advanced by an oil-driven micromanipulator (MO-97A, Narishige). The electric signal from the electrode was amplified with a bandpass filter (200 Hz-10 kHz; BAK). Neuronal spikes were isolated on-line using a custom voltage-time window discrimination software (MEX, LSR/NEI/NIH).

Histological procedure.

After the end of some recording sessions, we made electrolytic microlesions at the recording sites (15 μA and 30 s) in which we encountered typical Type 2 neurons in the mBF. After the conclusion of experiments, the monkey was deeply anesthetized using sodium pentobarbital and perfused with 10% formaldehyde. The brain was blocked and equilibrated with 10% sucrose. Frozen sections were cut every 50 μm in the coronal plane. The sections were stained with cresyl-violet.

Data processing and statistics.

Spike-density functions were generated by convolving spike-times with a Gaussian filter (σ = 50 ms). The conditioned stimulus (CS) neuronal responses were measured during a window of 150 ms after CS presentation until the end of the CS epoch. To normalize task-event related responses (Experiments 2–5), we subtracted baseline activity (last second of the intertrial interval) from the activity during the task-event related measurement epoch. All statistical tests were two-tailed. For comparisons between two task conditions for each neuron, we used rank-sum test, unless otherwise noted. For comparisons between two task conditions across the population average we used a paired signed-rank test, unless otherwise noted.

Experiment 1: appetitive and aversive Pavlovian procedure.

For this experiment, we recorded widely in the septal-basal forebrain complex in Monkeys P and Sm. To confirm our results in Monkeys P and Sm, we included data from two additional monkeys (T and S) used in a previous study (Monosov and Hikosaka, 2013).

The procedure consisted of two alternating blocks: appetitive and aversive blocks (Fig. 1). In the appetitive block, each of three CSs were followed by an unconditioned stimulus (US) (apple juice), with 100%, 50%, and 0% chance, respectively. In the aversive block, three CSs were followed by an airpuff US directed at the monkey's face with 100%, 50%, and 0% chance, respectively. The 0.4 ml of apple juice was delivered through a spout that was positioned in front of the monkey's mouth. Airpuff (∼35 psi) was delivered through a narrow tube placed 6–8 cm from the monkey's face.

Figure 1.

Responses of Type 2 neurons in the mBF to predictions about rewards and punishments. A, Monkeys experienced two distinct blocks: an appetitive block in which three visual CSs predicted juice (US) with 100%, 50%, and 0% chance, and an aversive block in which three cues predicted airpuff (US) with 100%, 50%, and 0% chance. TS, Trial start cue. B, Responses of a single Type 2 neuron in two blocks. Its spike activity is shown by raster plots (top) and spike density function (bottom). Dark blue raster plots indicate the activity in 50% CS trials in which US was omitted.

Each trial started with the presentation of a trial-start cue (white circle) at the center of the screen. After 1 s, the trial-start cue disappeared and one of the three CSs was presented pseudorandomly at the center. After 1.5 s, the CS disappeared, and the US (if scheduled for that trial) was delivered. The monkeys were not required to fixate the trial-start cue or CS. For Monkeys P and Sm, one block consisted of 12 trials with fixed proportions of trial types (100%, 4 trials; 50%, 4 trials; 0%, 4 trials), and the intertrial intervals (ITIs) ranged from 3 to 8 s. For Monkeys S and T, one block consisted of 22 trials, and the ITIs ranged from 5 to 10 s.

Neuron's uncertainty-preference was defined in Experiment 1 if its responses varied across the three CSs in either appetitive or aversive block (Kruskal–Wallis test, p < 0.05) and if its response to the uncertain CS (50%) was significantly stronger than its responses to both of the certain CSs (100% and 0%) (post hoc test: two-tailed rank-sum test; p < 0.01). To assess the heterogeneity of uncertainty sensitive neurons, we performed principal component analysis Figure 2). To quantify whether a neuron had a significant airpuff response, we compared the activity in a 50 ms time window starting from 20 ms after airpuff delivery (or omission) with the activity in a 50 ms time window before airpuff delivery or omission.

Figure 2.

Clustering of two groups of reward uncertainty-sensitive neurons. A, The CS epoch activity for 70 uncertainty selective neurons during the appetitive/aversive procedure. Each trace represents the average activity of a single neuron during the 6 CS epochs. We normalized each trace by dividing each of these 6 CS responses by the maximum CS response. PCA was performed on those normalized response functions. Orange and red traces represent the two clusters of neurons (Type 1 and Type 2) that resulted from the PCA analysis (shown in B, C). B, PCA performed on the single neuron functions in A revealed that the majority of the variance in the 70 neurons' CS activations can be explained by the first principal component (PC1). To assess this, we shuffled normalized activity across the 0% juice, 100% juice, 0% airpuff, 50% airpuff, and 100% airpuff CSs and calculated the percentage of variance explained (PVE) by each PC. Gray error bars represent the range of PVE for each PC based on 10,000 shuffles. C, Bimodal distribution of PC1s of the 70 uncertainty neurons (Hartigan's Dip Test, p < 0.01, tested by 10,000 permutations). Based on the PC1s, k-means clustering separated the neurons into two groups: orange represents Type 1; red represents Type 2. Their average firing is shown in D for the three CSs in each block (appetitive or aversive), separately.

For each neuron, the presence of ramping (Fig. 3D) was assessed with a correlation of neuronal activity (sampled every 100 ms) and time during the last 1 s of the CS period. Ramping activity was considered present if the correlation was positive and significant. The statistical significance of the correlation (p < 0.05) was tested using a permutation test (null hypothesis: neuronal activity was independent of time) by shuffling the neural firing rates across the time bins 2000 times. The slope of the anticipatory ramping activity was defined as the slope of the regression line fit to these data.

Figure 3.

Locations and firing properties of two groups of reward uncertainty-sensitive neurons in the septal area. A, B, Their locations are shown on MR images: left, coronal; right, parasagittal. Yellow represents Type 1 neurons in the anterodorsal septal region (n = 31). Red represents Type 2 neurons in the mBF (n = 39). Light green represents other recorded neurons (n = 300). C, Electrolytic marking lesions (black arrows) made at the locations of Type 2 neurons in the mBF of Monkey H. They were made along two mediolaterally adjacent electrode tracks. Along the lateral track, we recorded four Type 2 neurons between the two marking lesions. D, A table of response properties of Type 1 and 2 neurons recorded during Experiment 1. Asterisks indicate significant difference between Type 1 and Type 2 neurons (baseline firing rate and irregularity index; Wilcoxon rank sum test, p < 0.05; proportions of neurons displaying significant CSs ramping or airpuff responses; see Materials and Methods; χ² test, p < 0.05). A–C, Gray lines indicate 5 mm scale bars. A, C, The coronal sections are located ∼1.5 mm anterior to the center of the anterior commissure (ac). cc, Corpus callosum; CD, caudate nucleus; DBB, diagonal band of Broca; lv, lateral ventricle; LS, lateral septum; MS, medial septum; PUT, putamen; vmPFC, ventromedial prefrontal cortex.

The intrinsic firing characteristics of Type 2 neurons were defined as firing irregularity, autocorrelation width, and firing rate (Fig. 4). To quantify irregularity of spiking activity, we used the irregularity index (IR) developed by Davies et al. (Davies et al., 2006; Nakamura et al., 2008; Matsumoto and Hikosaka, 2009).This measurement does not require constant firing during the measurement period and is therefore useful for analyzing neurons with different firing properties. To verify our IR-based result, we also quantified spike rate variance, defined as the SD of intertrial intervals (ISIs) during the same baseline window. Neuronal burstiness was assessed during the same baseline window for each neuron using two measures previously used to study hippocampal burst neurons (Viskontas et al., 2007): (1) the width of spike autocorrelation (computed in 1 ms bins) defined as the first time (lag) the autocorrelation reached its mean point; and (2) the ratio of the number of ISIs <10 ms and the number of ISIs >10 ms.

Figure 4.

Properties of Type 2 neurons in mBF. A, Activity of 39 Type 2 neurons classified by PCA analysis (Fig. 2) during 100%, 50%, and 0% predictions of rewards and punishments. For each neuron, the activity was normalized by dividing the trial firing rate by the baseline firing rate. B, Proportion of Type 2 neurons (left column) and other mBF neurons (right column) displaying aversive signals (row 1), airpuff response (activity after airpuff > greater than activity before airpuff; row 2), and reward magnitude signals (row 3). Proportion of neurons displaying all three types of signals (rows 1–3) is shown in row 4. The proportion of Type 2 neurons displaying all three activity biases (64%) was significantly higher than the random combination (random combination = 12.5% binomial test; p < 0.0.01). C–F, Baseline firing characteristics of all Type 2 neurons recorded during all experiments in this study compared with other neurons in the mBF. Bar plots represent means. Single red dots indicate single neuron data. C, Firing irregularity indices. D, Autocorrelation width. E, Spike wave trough-to-trough duration. F, Baseline firing rate. *p < 0.05 (Wilcoxon rank sum test). ns, Not significant.

Experiment 2: unexpected reward and punishment delivery.

The procedure was the same as in Experiment 1, except that uncued trials were included in which a juice reward alone or an airpuff alone was delivered unexpectedly during some of the ITIs. These unexpected outcomes were included during 20% of the ITIs, which were chosen randomly. When the outcome was delivered, it occurred between ∼1.3 and 3 s after the end of the previous trial. Monkeys H and P participated in this experiment.

To compare the average reward or punishment responses after 50% CSs with average responses for the same outcomes delivered during the ITI (Fig. 5C,E), a paired signed rank test was performed comparing the activity in a 150 ms window after outcome delivery. To calculate and display the outcome response after the 50% CSs, the activity during 50% trials in which the outcome was not delivered was subtracted from 50% trials in which the outcome was delivered. To calculate and display the outcome response activity during the ITI, average baseline was subtracted.

Figure 5.

Response of single mBF Type 2 neurons to rewards and punishments. A, Activity of single Type 2 neurons during 100%, 50%, and 0% predictions of rewards and punishments (n = 14: 5 from Monkey H, 9 from Monkey P). For each neuron, the activity was normalized by dividing the trial firing rate by the baseline firing rate. B, Their average activity in the appetitive block. Conventions are the same as in Figure 1. C, Left, Difference in activity during trials in which rewards were delivered and not delivered after the 50% CS epoch (B, red vs pink trace). Single neuron activity (black) and average (blue) are shown. Right, Baseline-subtracted activity of the same neurons to unexpected rewards during the ITI. *p < 0.05, significant difference between the two conditions (left vs right; paired signed rank test). D, Average activity in the aversive block. E, Left, Difference in activity during trials in which airpuffs were delivered and not delivered after the 50% CS epoch. Right, Baseline-subtracted activity to unexpected airpuffs during the ITI. Same conventions as in C. Inset, Difference between less predicable airpuffs (after 50% CS epoch) and fully predictable airpuff responses (after 100% CS epoch).

Experiment 3: reward probability and reward amount Pavlovian procedure.

The procedure consisted of two blocks: a reward probability block and a reward amount block (Fig. 6). Monkeys H and P participated in this experiment.

Figure 6.

Differential sensitivities of mBF Type 2 neurons to reward amount and probability. A, Average activity of 31 mBF Type 2 neurons (15 in Monkey H, 16 in Monkey P) in a reward probability block (top) and a reward amount block (bottom). In the reward amount block, five fractal CSs indicated five different amounts of juice. In the reward probability block, five other fractal CSs indicated five different probabilities of juice. Expected values of the CSs in the two blocks were matched. TS, Trial-start cue; US, juice delivery. B, Average normalized CS responses of the same neurons for reward probability and reward amount CSs. Right, CS responses of single neurons: probability (black), amount (gray); normalized to the maximum CS response, from 0 to 1. C, Probability-amount difference in the activity of the mBF Type 2 neurons. The subtraction was done between a probability CS and an amount CS with an equal value. D, Normalized probability-amount CS responses. *p < 0.05 (paired sign-rank test). ns, Not significant. Error bars indicate SE.

In the reward probability block, five CSs were followed by a liquid reward (0.4 ml of juice) with 100%, 75%, 50%, 25%, and 0% chance, respectively. In the reward amount block, five CSs were followed by a liquid reward of 0.4, 0.3, 0.2, 0.1, and 0 ml, respectively. Thus, the expected values of the five CSs matched between the probability and amount blocks. Each trial started with the presentation of a trial-start cue at the center. The trial-start cue was a purple square in the probability block and a yellow square in the amount block. The monkeys had to maintain fixation on the trial-start cue for 1 s; then the trial-start cue disappeared and one of the five CSs was presented pseudorandomly. After 1.5 s, the CS disappeared, and juice (if scheduled for that trial) was delivered. The monkeys were not required to fixate the CSs. In each trial, the CS could appear in three locations: 10 degrees to the left or to the right of the trial-start cue, or in the center. One block consisted of 40 trials with fixed proportions of trial types (each of the five CSs appears eight times each block).

In a separate experimental session, we tested the monkeys' choice preference for the CSs (Fig. 7). The preference was tested in three ways: among the probabilistic CSs, among the amount CSs, and across the probabilistic and amount CSs. Monkeys H and P participated in this experiment. Each trial started with the presentation of the trial-start cue at the center, and the monkeys had to fixate it for 1 s. Then two CSs appeared,10 degrees to the left and right. The monkey had to make a saccade to one of the two CSs within 5 s and fixate it for at least 800 ms. Then the unchosen CS disappeared, and after ∼750 ms the outcome (associated with the chosen CS) was delivered and the chosen CS disappeared. If the monkey failed to fixate one of the CSs, the trial was aborted and all stimuli disappeared. The trials were presented pseudorandomly, so that a block of 180 trials contained all possible combinations of CSs four times.

Figure 7.

Monkeys' choice preference between CSs associated with different reward amounts and different reward probabilities. The monkey made a choice between two CSs among the 10 well-learned CSs (5 indicating reward amounts, 5 indicating reward probabilities) (Fig. 6). A, Choice between two reward probability CSs (left) and between two reward amount CSs (right). Each number indicates the choice percentage of the higher value CS. B, Choice percentage of a single reward probability CS versus all the other reward probability CSs (red). Choice percentage of a single reward amount CS versus all the other reward amount CSs (black). Data are compiled from a dataset of 3417 choice trials performed by Monkeys H and P.

Experiment 4: reward variance Pavlovian procedure.

The procedure consisted of one block in which four fractals were presented as CSs (Fig. 8). Monkeys H and P participated in this procedure. The “no variance” CS was always followed by 0.15 ml of juice. The “low variance” CS was followed either by 0.2 or 0.1 ml of juice, with a 50% chance. The “medium variance” CS was followed either by 0.25 or 0.05 ml of juice, with a 50% chance. The “high variance” CS was followed either by 0.3 ml of juice or no juice, with a 50% chance. Thus, the four CSs had the same expected value but different reward variances.

Figure 8.

Sensitivity of Type 2 neurons to reward variance. A, Experimental design. Four fractal CSs indicated four different levels of reward variance. Possible reward amounts are indicated with asterisks above each fractal. The large and small reward amounts occurred equally likely. The expected value was the same across the CSs. S.D, SD of reward amount. B, Average activity of 29 mBF Type 2 neurons (16 in Monkey H, 13 in Monkey P), shown separately for each of the four fractals. C, CS responses shown individually for the 29 neurons (normalized to the maximum CS response; from 0 to 1). D, Average CS responses of the same neurons. *p < 0.05 (paired sign-rank test). Error bars indicate SE.

Each trial started with the presentation of a trial-start cue (gray square) at the center. The monkeys had to maintain fixation on the trial-start cue for 1 s; then the trial-start cue disappeared and one of the four CSs was presented at the center. After 1.5 s, the CS disappeared, and juice (if scheduled for that trial) was delivered.

In a separate experimental session, we tested the monkeys' choice preference for the CSs (Fig. 9). Monkeys H and P participated in this task. The details are the same as in the choice procedure used in Experiment 3 (above).

Figure 9.

Monkeys' choice preference between CSs associated with different reward variances. The monkey made a choice between two CSs among the four well-learned CSs, indicating different reward variances (Fig. 8). A, Choice between two reward variance CSs. B, Choice percentage of a single reward variance CS versus all the other reward variance CSs. Data are compiled from a dataset of 4672 choice trials performed by Monkeys H and P.

Experiment 5: reward probability Pavlovian procedure with novel stimuli.

The procedure was the same as the appetitive block in Experiment 1, except that novel fractals were used for the CSs (Fig. 10). Also, here the monkey (P) was expected to fixate the trial-start cue for 1 s (as in Experiments 1–4). The monkey started learning some sets of CSs (three novel fractals for each set) in the Pavlovian procedure on consecutive days, one session per day. On the second and third days, more sets of CSs were added each day for the monkey's additional learning. Neuronal recording was done on the third day while the monkey was performing the Pavlovian procedure with these sets of CSs. In other words, the monkey had experienced the first group of CS sets for 2 d, the second group for 1 d, and the third group was novel. In addition, a well-learned set of CSs was used during the neuronal recording. This across-day procedure was done several times by introducing more novel fractals as CSs.

Figure 10.

Changing activity of Type 2 neurons during uncertainty resolution. Left and center columns, As Monkey P experienced new sets of fractal CSs on the appetitive task (Fig. 1A, left) across 3 d (one session each day), the activity pattern of mBF Type 2 neurons changed nonuniformly in time. Before the neuronal recording, the monkey had experienced multiple sets of fractal CSs with different degrees (e.g., 0, 1, or 2 d), which enabled the comparison of the same mBF neuron's activity across different degrees of learning. Right column, The monkey also had experienced a set of fractal CSs with many daily sessions (>100), which revealed the same neuron's responses to the well-learned CSs.

Choice trials (as a block of nine trials) were regularly inserted to the Pavlovian procedure (Fig. 11). On each trial, two of the three CSs were chosen and presented on the right and left (see Experiment 3). The purpose of the choice trials was to examine the changes in the monkey's behavior: choice preference, the number of saccades, and the time before making the choice.

Figure 11.

Time courses of neuronal and behavioral changes during uncertainty resolution. A, For each of the first, second, and third day of training (Day 1–3), average responses of Type 2 neurons to 100%, 50%, and 0% CSs are shown at the start and end of training (left and center columns). Their responses to well-learned CSs are shown for comparison (right column). *p < 0.05. ns, Not significant. Red square represents the first time when data for the newly learned CSs became statistically not different from data for the well-learned CSs. B, Behavioral changes during choice trials: choice probability of the higher valued CSs (top), target acquisition time during the choice (middle), and the number of saccades before choice (bottom). The behavioral data are shown for choice between 100% and 50% CSs, given the slower changes in neuronal responses between 100% and 50% CSs (A). Error bars indicate SE. C, Neuronal learning expressed as changes in differential responses between the uncertain forced single stimulus (50%) CS trials and the 100% CS trials during Day 1 (first learning session) quantified by ROC (ROCs > 0.5 indicate uncertainty selectivity). Type 2 neuron selectivity is plotted in black. For contrast, we recorded additional Type 1 ADS neurons during the same learning task (shown in red; 3 in Monkey Sm and 5 in Monkey P). As shown in A, Type 2 neurons learned slowly, whereas Type 1 neurons quickly displayed uncertainty preference as shown previously (Monosov and Hikosaka, 2013, their Fig. 5).

Experiment 6: manganese enhanced MRI tracing of mBF projections.

To test where the mBF Type 2 hotspot most strongly projects, we used manganese-enhanced MRI tracing (MEMRI) in Monkey P (Fig. 12). The method relies on two properties of manganese ions: (1) the manganese ion (Mn²⁺) is a calcium ion analog and is therefore taken up by neurons and transported in an anterograde fashion; and (2) Mn²⁺ increases the MR intensity of voxels (Pautler et al., 1998; Saleem et al., 2002; Simmons et al., 2008). Therefore, by injecting Mn²⁺ directly in to the brain, we can trace neuronal pathways using standard T1-weighted MRI.

Figure 12.

Transport of manganese chloride to the HF. A, The location of the injection site is visualized as a bright spot on sagittal (two images on the left) and coronal (right) MR images. Type 2 neurons recorded in the same monkey (classified by PCA analysis in Fig. 2) are shown in red (A, left). The 95% confidence ellipse around the neurons is shown in yellow. The same image is also shown without the neurons to observe the injection site within the confidence ellipse. White lines indicate 5 mm. CD, Caudate nucleus; PUT, putamen; vmPF, ventromedial prefrontal cortex. B, Percentage increase of voxel intensity (“MEMRI labeling”) shown in a parasagittal plane 24 h (B) and 96 h (C) after the manganese chloride injection. aHF, Anterior HF; pHF, posterior HF. D–F, MEMRI labeling shown in a coronal plane 24 h (E) and 96 h (F) after the injection. The coronal plane is focused on the HF whose structure is shown in D. CA fields are indicated within the hippocampus. TF, Parahippocampal cortical area TF. B, White line indicates 10 mm. D, White line indicates 5 mm. B, Red line indicates the location of the coronal plane shown in D–F.

To locate the injection site, we recorded neuronal activity within the mBF before the injection and verified that the neurons there were Type 2 neurons. We used a custom-made injectrode consisting of an epoxy-coated tungsten microelectrode for neuron recording and a silica tube for Mn²⁺ injection. Before the injection, the injectrode was placed at the region of the highest concentration of Type 2 neurons and the monkey performed the appetitive/aversive procedure (Fig. 1A). We injected 0.2 μl of 150 mm solution of manganese chloride in to the Type 2 hotspot within the mBF. This concentration was previously shown to be effective and nontoxic (Simmons et al., 2008; Eschenko et al., 2010). We made the injection in to the mBF over the duration of ∼20 min and waited for 15 min to retract. After this, we retracted the injectrode 0.5 mm and waited for 15 min, after which we retracted the injectrode from the brain and prepared the monkey for MRI.

MR anatomical images were acquired in a 4.7 T horizontal scanner (Bruker Biospec 47/40) using a modified driven equilibrium Fourier transform (MDEFT). The monkey's head was placed in the scanner in stereotactic position. A single loop circular coil was place on top of the animal's head. To minimize changes in RF across scans, we attached the MR surface coil directly to the monkey's head holder (in the head implant). The whole-brain MDEFT images were acquired in a 3D volume with a field of view 96 × 96 × 70 mm³, and 0.5 mm isotropic voxel size. The read-out had an 11 ms repetition time, a 4.1 ms echo time, and a 11.6 degree flip angle. The MDEFT preparation had a 1240 ms preinversion time, and a 960 ms postinversion time for optimized T1 contrast at 4.7 T. Each 3D volume took 25.5 min to acquire without averaging, and 51 min if average by two.

Three baseline preinjection scans were collected a week earlier. Postinjection scans were collected 24, 48, and 96 h after the injection. All scans were obtained using the averaged 51 min acquisition. Those scans were performed under isoflurane gas anesthesia, and 3–5 scans were collected in each session. On the injection day, three 25 min scans were collected 1 h after the injection was finished under ketamine anesthesia to localize the injection site.

As in previous monkey MEMRI experiments, we could not detect manganese transport by eye 24 h after the injection (Simmons et al., 2008). To visualize the transport, we calculated the percentage increase of voxel intensity after the injection by comparing averaged preinjection scan with averaged postinjection scans. Before doing this, the averaged MRIs were processed by the Analysis of Functional NeuroImages (AFNI) toolkit (Cox, 1996). First, the image nonuniformity was reduced (AFNI function: 3dUniformize). Second, the images were resampled, doubling the number of voxels (AFNI function: 3dresample). Third, the postinjection scans were aligned with the average preinjection baseline scan using affine transforms (AFNI function: 3dAllineate). Fourth, the alignment of postinjection scans was further refined using a nonlinear warping algorithm (AFNI function: 3dQwarp). Fifth, the averaged images (pre and post) were smoothed with a 3D Gaussian filter (σ = 0.2 mm). Last, before calculating the percentage increase, each average scan was normalized by dividing by the average intensity value of 5 mm³ of cortex.

Results

Septal-basal forebrain complex contains distinct populations of neurons sensitive to reward uncertainty (Experiment 1)

We defined the mBF as a region containing the medial septum and the nucleus of the diagonal band of Broca. We recorded activity of single neurons in the mBF and surrounding septal regions while monkeys participated in a Pavlovian procedure in which appetitive and aversive blocks were alternated (Fig. 1A). In the appetitive block, three visual CSs predicted juice with 100%, 50%, and 0% chance. In the aversive block, three CSs predicted airpuffs with 100%, 50%, and 0% chance. Four monkeys (Monkeys P, S, Sm, and T) participated in this procedure.

We found a group of neurons that showed reward uncertainty-selective activity. An example is shown in Figure 1B. In the appetitive block, the neuron displayed a monotonic increase in firing rate during the uncertain (50% juice) CS period. The activity was truncated in response to the outcome (i.e., delivery or omission of juice). In contrast, the neuron showed no increase in activity during the certain (100% and 0% juice) CS periods. The neuron was also sensitive to the uncertain CS in the aversive block, but not selectively. It displayed similar ramping activity during the uncertain (50% airpuff) and certain (100% airpuff) CS periods. The neuron also showed a phasic response to the delivery of airpuff.

Among 370 neurons recorded in the septal-mBF complex, 70 neurons showed reward uncertainty-selective activity. To quantitatively test the heterogeneity among the reward uncertainty neurons, we performed principal component analysis (PCA) using the magnitudes of the neuron's responses to the 6 CSs in the appetitive and aversive procedure (Fig. 2). The PCA analysis showed that the reward uncertainty neurons were categorized into two distinct types: Type 1 (n = 31) and Type 2 (n = 39). Type 1 and 2 neurons were different in several aspects: anatomical locations within the septal-basal forebrain complex, the pattern of CS and US responses, background firing rates, and other electrophysiological features (Fig. 3). The example neuron (Fig. 1B) belonged to Type 2 population. Notably, we found no punishment uncertainty neurons (i.e., respond more strongly to 50% airpuff CS than 100% or 0% airpuff CS) in the septal-mBF complex.

We found that Type 1 and 2 neurons were located largely separately in the septal-mBF complex (Fig. 3). Most Type 1 neurons (yellow dots) were located in the anterodorsal septum (ADS). They correspond to a group of reward uncertainty neurons reported previously (Monosov and Hikosaka, 2013). In contrast, most Type 2 neurons (red dots) were located in the mBF (medial septum and diagonal band of Broca). The locations of Type 2 neurons were confirmed histologically (Fig. 3C). A total of five Type 2 neurons were recorded at and between the marking lesions (arrows), which were made along two electrode tracks. They showed very similar activity to the neuron shown in Figure 1B. All of them were located in the diagonal band of Broca. In the rest of the manuscript, we will focus on Type 2 neurons located in the mBF.

Characterization of Type 2 neurons in the mBF

Unlike Type 1 neurons in the ADS, single Type 2 neurons in the mBF (hereafter called “mBF Type 2 neurons”) consistently displayed responses to information other than reward uncertainty. These signals were combined in a rather unique way. This is shown qualitatively in Figure 4A in which activity of 39 Type 2 neurons clustered by PCA analysis (Fig. 2) is superimposed. First, in addition to the ramping activity selective for reward uncertainty (50% juice CS), they showed similar ramping activity when airpuff was expected (100% or 50%; the slopes of the ramping for reward uncertainty and airpuff expectation were correlated across neurons; ρ = 0.69; p < 0.01). Second, they were phasically activated by airpuff. Third, in the appetitive block, their activity tended to be higher during 100% CS than 0% CS. A majority of reward uncertainty-sensitive neurons in the mBF (Type 2) showed the same types of activity bias (Fig. 4A,B). In particular, the proportion of neurons that combined the three activity biases (64%) was significantly higher than the random combination (random combination = 12.5% binomial test; p < 0.0.01). This combination of activity was not present among mBF neurons that were not sensitive to reward uncertainty (n = 211) (Fig. 4B). These data suggest that the reward uncertainty-sensitive neurons in the mBF comprise a group of neurons that were characterized by the unique combination of activity.

Based on that observation, we wondered whether Type 2 neurons' intrinsic firing properties set them apart from other mBF neurons. We found some differences (Fig. 4C–E). First, Type 2 neurons had lower firing irregularity indices than other mBF neurons (Fig. 4C; p < 0.05) and had lower variance of ISIs than other mBF neurons (p < 0.05). Second, Type 2 neurons' firing was less bursty than other mBF neurons, which was shown by two measurements (Viskontas et al., 2007): autocorrelation width (Fig. 4D; p < 0.05) and ISI ratio (p < 0.05; Materials and Methods). Third, Type 2 neurons had narrower spike durations than other mBF neurons (Fig. 4E; p < 0.05). Fourth, however, there was no significant difference in their firing rate (Fig. 4F).

mBF Type 2 neurons signal unexpected outcomes (Experiment 2)

In this and following experiments and analyses, we examined how mBF Type 2 neurons signal uncertainty and value, and how their responses change during learning. As shown before, mBF Type 2 neurons showed strong predictive activity before an uncertain outcome of reward but were insensitive to the outcome (Fig. 2D, bottom).

We found, however, that mBF Type 2 neurons responded strongly when the reward was delivered unexpectedly outside the task context (Fig. 5). In a separate set of experiment using the same Pavlovian procedure (Fig. 1A), we delivered juice or airpuff (with the same amount and intensity) occasionally and randomly during the ITIs. mBF Type 2 neurons responded phasically to both juice and airpuff (Fig. 5C,E, right). The results indicate that mBF Type 2 neurons signal a reward outcome only when it is delivered unexpectedly outside the behavioral context in which the monkey is currently engaged. The context dependency was less selective for aversive outcomes.

mBF Type 2 neurons combine reward uncertainty and value (Experiment 3)

While the neural selectivity for reward uncertainty was the most pronounced aspect of Type 2 neural activity, there was an additional asymmetry in the responses to the two certain conditions. As shown in Figure 2D (bottom), the certain reward condition (100%) gave rise to a larger response than the certain no-reward condition (0%) (p < 0.05).

To investigate this further, we recorded Type 2 activity using a Pavlovian procedure that contained two distinct contexts: a reward probability block and a reward amount block (Fig. 6). In the reward probability block, five CSs indicated five different probabilities (0%, 25%, 50%, 75%, 100%) of 0.4 ml of juice. In the reward amount block, five other CSs indicated five different amounts of juice (0, 0.1, 0.2, 0.3, 0.4 ml). Thus, the expected values (calculated as follows: amount × probability) of the CSs were matched between the two blocks.

In the reward amount block, the activity of mBF Type 2 neurons during the CS period varied with reward values, which were represented by reward amounts (Fig. 6A, bottom). Their activity increased nonlinearly as the value increased but nearly plateaued at the two highest levels (Fig. 6B, gray). Thus, in the absence of reward uncertainty, the activity of these neurons was consistently modulated by expected value of the CS.

However, in the reward probability block, where expected value and reward uncertainty covaried, activity reflected a combination of these variables. In anticipation of the reward, neural firing rate increased as the expected value of acquiring reward increased, but then decreased at the highest levels where the uncertainty went to zero (Fig. 6B, black). Overall, the activity of mBF Type 2 neurons was higher in the reward probability block than in the reward amount block, specifically when the CS indicated reward uncertainty. As in Experiment 1, the anticipatory increases in activity were mostly observed in response to uncertain reward predictions, not changes in certain reward.

The data thus far suggest that BF Type 2 neurons encode the combination of value and uncertainty. If so, the difference in activity between the probability and amount blocks would show their sensitivity to reward uncertainty. Indeed, as shown in Figure 6C, the subtracted activity grew larger monotonically during the CS period only when reward was uncertain. It was highest when the reward probability was 50% than 25% or 75%. Overall, the subtracted activity during the CS period showed an inverse U-shaped pattern (Fig. 6D), matching previous findings of Type 1 neurons in the ADS (Monosov and Hikosaka, 2013). This combination of results raises the intriguing possibility that BF Type 2 neurons' activity reflects a specific convergence of neural inputs. Type 2 neurons may be the recipient of uncertainty-selective signal from ADS Type 1 neurons as well as a value-selective signal from other neurons located elsewhere in the brain. A straightforward additive combination of these two signals would account for the complex properties observed in the Type 2 neurons, particularly because the subtracted activity (Fig. 6C) showed no hint of a value signal (Fig. 6A, bottom).

The stronger responses of BF Type 2 neurons to uncertain CSs might be related to the monkey's preference of reward uncertainty. However, this possibility was not supported by the following experiment: We presented two CSs simultaneously and let the monkey choose one. The monkey's choice was largely guided by the expected reward value in any of the following conditions: when the choice was made between two reward probability CSs or between two reward-size CSs (Fig. 7).

Alternatively, the stronger responses of BF Type 2 neurons to uncertain CSs might be caused by their response selectivity to visual objects. To test this possibility, we used two sets of CSs for 11 of the 31 Type 2 neurons recorded in Experiment 3. We found that their responses were similar for both sets (p > 0.1), suggesting that the responses of BF Type 2 neurons were sensitive to reward uncertainty, not visual object identity.

mBF Type 2 neurons signal reward variance (Experiment 4)

In the probability block of Experiment 3, changes in uncertainty were always associated with changes in value because the reward amount was held constant. If uncertainty and value signals in BF Type 2 neurons have distinct origins, then those neurons should encode the quantitative level of uncertainty when the value is held constant.

We tested this hypothesis using a Pavlovian procedure in which we varied the variance of reward amount while the expected value remained unchanged across all the trials (Fig. 8A). Among four CSs used, one was a certain CS and the other three were uncertain CSs. Each of the uncertain CSs predicted two amounts of juice with an equal probability (i.e., 50%). Across the CSs, the average of the two amounts remained the same, but the difference between the two amounts varied. The difference (or variance) is often taken as a measure of risk (Weber et al., 2004; Preuschoff and Bossaerts, 2007; Platt and Huettel, 2008; Bach and Dolan, 2012).

We found that BF Type 2 neurons encoded reward variance (or risk) (Fig. 8B–D). Their average activity increased monotonically during the trial-start cue and CS period until the resolution of uncertainty (Fig. 8B). Importantly, the increase in activity during the CS period was higher when the reward variance was higher (Fig. 8D). This tendency was common among 29 BF neurons tested (Fig. 8C).

Unlike Experiment 3, all the CSs were associated with the same expected reward value, but with different levels of reward variance. When we presented two of the four CSs (Fig. 8A) simultaneously, the monkeys tended to choose the higher-variance (i.e., riskier) CS (Fig. 9). The result confirmed previous findings (McCoy and Platt, 2005; Platt and Huettel, 2008; O'Neill and Schultz, 2010; So and Stuphorn, 2010).

mBF Type 2 neurons slowly acquire reward uncertainty signal (Experiment 5)

Experiments 1–4 were performed after the monkeys had learned the meanings of all CSs: the associated values and the degree of uncertainty. However, when the monkeys experienced the CSs for the first time, the meaning of each CS was unknown. How do BF Type 2 neurons acquire their sensitivity to value and uncertainty?

To answer this question, we tested the same type of neurons (i.e., Type 2) across daily sessions using CSs with different degrees of learning (Fig. 10). The task was the same as used in Experiment 1 but included only the appetitive block. In each session, we used three sets of objects as CSs: (1) a new set of objects, (2) a not well-learned set of objects (learned once or twice), and (3) a well-learned set of objects (learned > 1 month).

In response to new CSs (Fig. 10, Day 1, left), these Type 2 neurons increased their activity monotonically for all CSs until the delivery of an outcome (i.e., juice or no juice). Their responses changed by the end of the 1 day learning session (Fig. 10, Day 1, middle): the 0% CS elicited a strong suppression (p < 0.05). Thus, after 1 d learning, the response to the 0% CS reached the level of the well-learned response (Fig. 11A, bottom row). In contrast, their responses to the 100% and 50% CS remained unchanged (Figs. 10, 11A, top and middle rows). This response pattern was similar to their value-coding (Fig. 6A, bottom), except that their activity ramped up toward the US for all CSs.

It took ∼2 days for the neurons to develop their uncertainty selectivity (Fig. 10, middle row). This occurred as the BF Type 2 neurons decreased their responses to 100% CS, so that the responses to 50% CS became larger than both the responses to 0% CS and 100% CS (p < 0.05; paired signed rank test). By the end of the 2 day learning session, their responses to all CSs were not significantly different from their responses to well-learned CSs (Fig. 11A, center column). However, the responses to 100% CS appeared less suppressed even after the 2 day or 3 day learning session than after the long-term learning. In summary, the slow time course of uncertainty selectivity was due to a gradual decrease in the magnitude of the 100% CS responses.

How might the changes in the activity of BF Type 2 neurons be related to the monkey's behavior? To investigate this issue, after every 12 single CS trials, the monkey experienced nine choice trials. During these trials, the monkey chose one CS among two by fixating it for 800 ms. The monkey was free to look around at the CSs before making his choice for up to 5 s. The details of this procedure are explained in Materials and Methods.

By the end of the 1 day learning session, the monkeys already chose the higher-valued CS on most trials, as they do for the well-learned CSs (Fig. 11B, left column, top). Correspondingly, BF Type 2 neurons appeared to have acquired value signals by the end of the 1 day learning session (see above). With further learning, however, the monkey's behavior continued to change: decision speed became faster. This was shown as a continued decrease in the target acquisition time and the number of saccades made before the target acquisition (Fig. 11B, middle and bottom rows). Only by the end of the 2 day learning session, these values became indistinguishable from those for well-learned CSs. These data suggest that: (1) long-term learning is required for the ramping activity of BF Type 2 neurons to be tuned to the uncertainty of reward outcome; and (2) this process might be related to the monkey's confidence or skill to choose high-valued objects.

It is noteworthy that the reward uncertainty activity developed at different speeds between Type 2 neurons in the mBF and Type 1 neurons in the ADS (Fig. 11C). Unlike Type 2 neurons, Type 1 neurons developed uncertainty selectivity within one session on the first day of learning, as we previously showed (Monosov and Hikosaka, 2013). The data provide another distinct feature between Type 1 and Type 2 neurons.

Type 2 hotspot projects to the intermediate HF (Experiment 6)

The function of mBF Type 2 neurons depends on where their signals are sent via anatomical projections. To address this issue for the population of mBF neurons we recorded, we used MEMRI. As manganese is taken up and transported anterogradely by neurons and is also visible in MRI scans, this method allows for in vivo tracing of neuronal connections in monkeys (Saleem et al., 2002; Simmons et al., 2008) and then allows the subject to continue to participate in tasks/experiments after the tracing procedure is completed. Using electrophysiologically guided positioning of an injection cannula, we introduced manganese chloride solution (MnCl₂, 0.2 μl, 150 mm) into the mBF area in which we found many Type 2 neurons (Fig. 12A). This can be qualitatively observed by comparing Figure 12A with Figure 3. To quantitatively verify this observation, we also generated a 95% confidence ellipse in MRI space from the locations of the Type 2 neurons recorded in the same monkey (Fig. 12A) and confirmed that the injection was within this ellipse (Fig. 12B).

We subsequently acquired a series of high-resolution MRI scans across a period of ∼100 h to visualize how manganese was transported (Materials and Methods).

Twenty-four hours after the injection, we found MEMRI labeling most prominently in the intermediate region of the ipsilateral HF (Fig. 12B,E), including the hippocampus and medial temporal cortices. The signal continued to increase over the course of days (Fig. 12C,F). The contralateral HF was also labeled similarly but more weakly (data not shown). No obvious labeling was found in other brain areas. These experiments suggest the possibility that the signals of mBF Type 2 neurons are transmitted to the HF, especially in its intermediate region.

Discussion

Within the mBF in the macaque monkey, a group of neurons encoding reward uncertainty (Type 2 neurons) also signaled a number of other internal variables related to motivation and expectancy (Fig. 4B). These responses occurred in two contexts: before or after an outcome. First, the preoutcome responses occurred when the monkey was engaged in a task, reflecting the monkey's prediction. The reward uncertainty response appeared as a monotonic increase in activity that was rapidly truncated after the outcome (reward or no reward). A similar monotonic increase of activity occurred when the monkey anticipated a certain or uncertain punishment. The reward value response appeared as a change in tonic firing (Fig. 6). Second, the postoutcome responses occurred strongly when an outcome was delivered unexpectedly. The unexpected nature was particularly clear for the postreward response (Fig. 5). In contrast, no postreward response occurred during task performance, even when the reward was uncertain (Fig. 2D, bottom), a situation that creates reward prediction error (Schultz et al., 1997; Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010). Therefore, BF Type 2 neurons encode “surprise” rewards (or strong/highly salient outcomes), but not reward prediction error. These results indicate that single mBF Type 2 neurons encoded a unique combination of information: reward uncertainty, expected reward value, punishment prediction, surprise rewards and punishments.

Our experiments suggest that the different kinds of information encoded by BF Type 2 neurons may originate from different sources. For example, BF Type 2 neurons combined information about “value” (or reward size) and information about uncertainty additively (Fig. 6). Also, value selectivity developed earlier during learning than uncertainty selectivity (Fig. 10). The distinct nature of value and uncertainty signals in BF Type 2 neurons may provide their target areas with a signal about behavioral relevance that is balanced between value and uncertainty and partly based on the subject's experience.

A likely source of the reward uncertainty signal in BF Type 2 neurons is the ADS where we found neurons that selectively encoded reward uncertainty (Monosov and Hikosaka, 2013). In this report, we classified them as Type 1 neurons, with their locations shown in Figure 3 and their activity shown in Figure 2 (together with Type 2 neurons). Notably, the inverse U-shaped uncertainty coding of BF Type 2 neurons (Fig. 6D) was very similar to that of ADS Type 1 neurons (Monosov and Hikosaka, 2013) (Fig. 3C). The connection from ADS Type 1 neurons to BF Type 2 neurons may be supported by some rodent anatomical studies (Meibach and Siegel, 1977; Russchen et al., 1985; Staiger and Nürnberger, 1991). Interestingly, Type 1 neurons developed the reward uncertainty response quickly in one learning session, whereas Type 2 neurons acquired reward uncertainty selectivity by eliminating their responses to certain CSs and did so slowly across days (Fig. 10). Therefore, it is unlikely that the reward uncertainty signal is directly transmitted from Type 1 to Type 2 neurons. Instead, a connection might act as a conditioning signal for Type 2 neurons such that their activity during the uncertain CS (not certain CSs) was preserved. In other words, the ADS could be, in effect, training the mBF to shape its responses based on uncertainty.

On the other hand, the sources of the other signals in BF Type 2 neurons are unknown. One candidate may be the medial thalamic nuclei (Hsu and Price, 2009), which receive dense inputs from the ventromedial prefrontal cortex (Hsu and Price, 2007, 2009) where distinct populations of neurons tonically encode appetitive or aversive states (Monosov and Hikosaka, 2012). Other candidates include the brainstem serotonergic and adrenergic nuclei (Russchen et al., 1985; Semba et al., 1988; Vertes, 1988; España and Berridge, 2006).

What then is the function of mBF Type 2 neurons? Lesion and brain imaging studies have linked the mBF to learning and memory functions (Damasio et al., 1985; Ridley et al., 1988, 1989, 1999a; Morris et al., 1992; Roman et al., 1993; Abe et al., 1998; De Rosa and Hasselmo, 2000; Fujii et al., 2002; De Rosa et al., 2004; Baxter and Bucci, 2013; Iglesias et al., 2013). Anatomically, the mBF is shown to project to brain areas that are heavily involved in learning and memory, including the HF and the anterior cingulate cortex (Mesulam et al., 1983). We thus consider the possibility that mBF Type 2 neurons serve learning and/or memory functions.

A fundamental question is as follows: Why do mBF Type 2 neurons encode a relatively fixed combination of information related to both rewards and punishments? Interestingly, single mBF Type 2 neurons encoded the combination of the reward-related and punishment-related information in a unidirectional manner: they increased their firing rate in reward-associated and punishment-associated contexts (Fig. 2D, bottom). Therefore, Type 2 signals would not be suitable to guide future actions aiming at obtaining rewards and avoiding punishments (e.g., through reinforcement learning). Consistent with Type 2 neurons not being a source of reinforcing signals are two additional observations: (1) Type 2 neurons did not encode reward prediction errors; and (2) they responded more strongly to 50% than 100% rewards. These data indicate that Type 2 neurons could not reliably guide behavior toward rewards in uncertain contexts.

On the other hand, the same combination of information would be suitable for creating or modifying episodic memories as well as for other monitoring functions (e.g., attention for learning). For instance, we remember past events more likely if they were associated with something rewarding, punishing, or surprising (Loewenstein et al., 2001; Dolcos et al., 2004a, b; Hulse et al., 2007). Consistent with this hypothesis, MEMRI tracing showed that the region of the mBF where Type 2 neurons were concentrated projected to the HF, which is a key region for forming and recalling episodic or abstract associative memories (Hasselmo et al., 1996; Mishkin et al., 1997; Vargha-Khadem et al., 1997; Doré et al., 1998; Wallenstein et al., 1998; Manns et al., 2003; Ergorul and Eichenbaum, 2004). The HF is known to receive continuous information about the external world from cerebral cortical areas and the internal state from neuromodulatory systems, including the dopaminergic system (Lisman and Grace, 2005). The signals from the mBF could instruct the HF to take a “snapshot” of the ongoing events and contexts when they are rewarding, punishing, or surprising. These “snapshots” of the world would be used as important evidence for the animal's judgment and decision making (Tulving, 2002; Eichenbaum, 2004; Bar, 2009; Squire and Wixted, 2011; Stokes et al., 2012).

Computational models have suggested that the mBF exerts control over learning and episodic memory in the HF (Hasselmo et al., 1995, 1996; Myers et al., 1996). In support of these models and our hypothesis, disruptions of the HF and mBF can produce similar deficits in memory formation (Ridley et al., 1988, 1989, 1999a; Morris et al., 1992; Roman et al., 1993; Suzuki et al., 1993, 1997; Abe et al., 1998; Turchi et al., 2005, 2008).

More experiments are now necessary to further elucidate the role of mBF Type 2 neurons in cognitive functions. Some studies have suggested that, in certain contexts, the HF contributes to the reduced attention to irrelevant events rather than the increased attention to relevant events (Han et al., 1995), which is different from what we proposed above (i.e., attention for learning). Notably, this alternative function of the HF may be guided by cholinergic neurons in the mBF (Chiba et al., 1995; Baxter et al., 1997; Oros et al., 2014). On the other hand, noncholinergic mBF neurons may be responsible for increasing learning and memory capacities (Dwyer et al., 2007; Pang et al., 2011; Baxter and Bucci, 2013; Roland et al., 2014). This consideration suggests that Type 2 neurons in the mBF are noncholinergic. Indeed, unlike Type 2 neurons (Fig. 4C–F), cholinergic neurons typically have low firing rates, wide spike durations, and display irregular firing (Manns et al., 2000a, b; Momiyama and Zaborszky, 2006; Simon et al., 2006; Hedrick and Waters, 2010; Unal et al., 2012). However, the neurochemical profile of Type 2 neurons is unknown. Although BF is often characterized by concentrated groups of cholinergic neurons (Mesulam et al., 1983; Zaborszky et al., 2008), it contains other kinds of neurons, including GABAergic and glutamatergic neurons (Semba, 2000; Lin et al., 2006; Zaborszky et al., 2008), all of which may project to the HF (Everitt and Robbins, 1997; Semba, 2000; Easton et al., 2012). Therefore, finding out whether Type 2 neurons belong to a particular neuronal group would be important to reveal the circuit mechanisms that control what to remember and from what to learn.