Skip to main content
PMC home page
Learning & Memory logoLink to Learning & Memory
. 2014 Feb;21(2):105–118. doi: 10.1101/lm.032433.113

Learning to ignore: A modeling study of a decremental cholinergic pathway and its influence on attention and learning

Nicolas Oros 1,4, Andrea A Chiba 3, Douglas A Nitz 3, Jeffrey L Krichmar 1,2
PMCID: PMC3895228  PMID: 24443744

Abstract

Learning to ignore irrelevant stimuli is essential to achieving efficient and fluid attention, and serves as the complement to increasing attention to relevant stimuli. The different cholinergic (ACh) subsystems within the basal forebrain regulate attention in distinct but complementary ways. ACh projections from the substantia innominata/nucleus basalis region (SI/nBM) to the neocortex are necessary to increase attention to relevant stimuli and have been well studied. Lesser known are ACh projections from the medial septum/vertical limb of the diagonal band (MS/VDB) to the hippocampus and the cingulate that are necessary to reduce attention to irrelevant stimuli. We developed a neural simulation to provide insight into how ACh can decrement attention using this distinct pathway from the MS/VDB. We tested the model in behavioral paradigms that require decremental attention. The model exhibits behavioral effects such as associative learning, latent inhibition, and persisting behavior. Lesioning the MS/VDB disrupts latent inhibition, and drastically increases perseverative behavior. Taken together, the model demonstrates that the ACh decremental pathway is necessary for appropriate learning and attention under dynamic circumstances and suggests a canonical neural architecture for decrementing attention.

In order to cope with uncertainty in the environment, organisms need to constantly regulate their attentional resources by increasing attention to salient or relevant cues, and also decreasing attention to irrelevant cues. The basal forebrain (BF) has specific and separate pathways for decrementing and incrementing attention. Whereas ACh projections from the medial septum/vertical limb of the diagonal band (MS/VDB) to the hippocampus and cingulate cortex are crucial to reduce attention to irrelevant stimuli (Baxter et al. 1997), ACh projections from the substantia innominata/nucleus basalis region (SI/nBM) to the neocortex are necessary to increase attention to relevant stimuli (Chiba et al. 1995). The importance of disregarding irrelevant information in the environment in support of fluid attention to relevant information is highlighted by learning difficulties in which this function is dysregulated, including various subtypes of attention deficit hyperactivity disorder, a form of mild cognitive impairment that precedes dementia, and schizophrenia (for review, see Lubow and Weiner 2010).

The decremental pathway from MS/VDB is necessary for decreasing attention to nonrewarding stimuli. Baxter et al. (1997) and Chiba et al. (1995) performed an experiment designed to test latent inhibition, which is the slower learning of a previously exposed, uninformative stimulus. Lesions of the MS/VDB, but not SI/nBM, disrupted latent inhibition. That is, control rats showed diminished conditioning to a preexposed stimulus, as compared to a novel stimulus. However, conditioning to the novel and preexposed cues occurred at the same level in the MS/VDB lesioned rats. The incremental pathway from SI/nBM is necessary for increasing attention in order to perform demanding associative learning. In an additional set of experiments conducted by Chiba et al. (1995) and Baxter et al. (1997), each rat was exposed to conditioned stimuli (CS) that were either consistent or inconsistent predictors of subsequent cues. Control rats showed increased CS associability when that cue was an inconsistent predictor of a subsequent cue, whereas rats with lesions to the SI/nBM were impaired in their ability to increase attention toward the CS when its established relation to another cue was modified. Similar to controls, rats with MS/VDB lesions showed increased CS associability. These results showed that ACh projections from the MS/VDB to the hippocampus and cingulate are important for decrementing attention, and ACh projections from the SI/nBM to the neocortex are important for incrementing attention (Chiba et al. 1995; Baxter et al. 1997). Baxter et al. (1997) also hypothesized that the role of the hippocampus in decremental processing might be similar to the role of the central amygdala in incremental processing. They might both regulate processing in cortical and subcortical targets via their efferent projections to the basal forebrain (Swanson 1977; Gaykema et al. 1990, 1991; Holland and Gallagher 1993; Gallagher and Holland 1994; Chiba et al. 1995).

Numerous studies have shown how ACh originating from the SI/nBM is involved in increasing attention. Disney et al. (2007) found that ACh was enhancing the detection of visual stimuli by increasing the gain of thalamic input to the visual cortex. Han et al. (1999) observed a role for the amygdala's central nucleus (CeA) projecting to the SI/nBM in incrementing conditioned stimulus processing in rats. Similarly, Holland (2007) found that projections from the CeA to the SI/nBM are necessary to increase attention enough for rats to perform a modified five-choice serial reaction time task. For reviews on how ACh can increases attention by sharpening relevant sensory inputs, see Hasselmo and McGaughy (2004), Sarter et al. (2005), and Disney et al. (2007). Although the role of ACh in increasing attention has been studied extensively, to the best of our knowledge only Baxter et al. (1997) showed a role for ACh from the MS/VDB in decreasing attention.

Therefore, the focus of this paper is on how ACh from the MS/VDB modulates the neural processing of a decremental pathway in order to decrease attention (Baxter et al. 1997). Following the experiments conducted by Chiba and Baxter, we define attention within an associative learning framework as a process that can regulate CS processing and action selection in the context of reinforcement learning. We investigate neuromodulatory influences on a specific pathway important for regulating attention during cognitive tasks. The role of ACh in regulating attention has been investigated by several computational models (Yu and Dayan 2005; Deco and Thiele 2011; Avery et al. 2012b). However, no modeling studies have investigated how ACh can decrement attention using a specific pathway, and have shown its role in tasks involving associative learning, latent inhibition, extinction, and reversal learning. In this paper, we use a relatively simple model (Fig. 1; see Materials and Methods) in order to suggest a mechanism in which the cholinergic decremental pathway can lead to behavioral effects commonly observed in animals, such as latent inhibition, extinction, and persisting behavior.

Figure 1.

Figure 1.

Open in a new tab

(A) Neural architecture consisting of an input area, a Decremental area, a cholinergic neuromodulatory system (MS/VDB), a Modulated input area, and an action selection area. Every neural area had two neurons responding to different input cues. Between different neural areas, these neurons were connected through “one-to-one” synaptic projections. The dashed lines represent inhibitory projections. An action was selected if the activity of a neuron in the Action selection area was above a certain threshold (0.5), otherwise a random choice was made. (B) Description of a trial. During each trial, the model was presented with two cues simultaneously for 10 time-steps (observation period). At the tenth time-step, the model made a selection. Once a cue was chosen, the model was presented with the selected cue for an additional 10 time-steps (action period).

Results

A neural network abstracting the known functional neuroanatomy for attentional pathways was constructed (Fig. 1; see Materials and Methods). It consisted of an Input area, a Decremental area (which loosely corresponds to the hippocampus and cingulate), a Modulated input area, an Action selection area, and one neuromodulatory cholinergic area (MS/VDB). In our model, we define the cholinergic decremental (attentional) pathway as the MS/VDB projections to the Decremental area. We investigated the role of this decremental pathway with separate simulated experiments targeting specific cognitive behaviors, such as first order associative learning, latent inhibition, extinction, and reversal learning. Simulated lesions of the cholinergic system (MS/VDB) were conducted during each experiment to elucidate the different roles of a decremental pathway.

During a typical trial, our model had to learn that only one cue out of two predicted a positive reward. On each trial, the model was first presented with two cues simultaneously during an observation period (see Fig. 1B) by setting the activation of both neurons of the Input area to one. The activation of these two units then propagated to both the Modulated input and the Decremental areas (as shown in Fig. 1A). Over time, the plastic connections between the Decremental area and the MS/VDB were strengthened, causing increased neural activity in MS/VDB. This triggered the release of ACh on synapses to the Decremental layer, resulting in amplification of the signal coming from the input layer. With this boost, the Decremental layer neurons strongly inhibited the corresponding units in the Modulated input layer. The main function of the ACh decremental pathway was therefore to regulate the Decremental area in order to cause a strong inhibition of the Modulated input area. The Modulated input area integrated excitation from input neurons with inhibition from Decremental neurons, and propagated activity into the Action selection area. An action was then selected if the activation of one of the units was >0.5 and was greater than that of the other unit. This resulted in a second round of activation propagation through the network during the second part of the trial (action period), in which only the selected unit was activated in the input layer. If, however, neither action was clearly selected during the first part of the trial, an action was chosen at random and the network was presented with the corresponding stimulus. The connections between the Modulated input and the Action selection areas were subject to a reward-dependent Hebbian learning rule, which increased the connection strength every time the model made a correct action and received a reward, and slowly decreased the connection strength otherwise. Over a number of trials, one connection became much stronger than the other, causing the model to select the rewarding stimulus and ignore the nonrewarding one. We have to emphasize that since the ACh decremental pathway could regulate the Decremental area in order to reduce the activity of the Modulated input area, it could hinder Hebbian learning indirectly. Therefore, the decremental pathway played an indirect but important role in reinforcement learning for the present experiments.

For all the experiments we conducted, a trial ran for 20 time-steps (10 for the observation period, 10 for the action period). One time-step in our simulation would correspond to ∼100 ms in real time. More modeling details can be found in the Materials and Methods section.

First order associative learning

Behavioral data

In the first order associative learning task, the model had to learn that one cue was predictive of a reward (see Fig. 2, Task 1). In this experiment, the model had not previously encountered the cues. The experiment continued until the model had learned and performed 10 consecutive correct choices (criterion). We recorded the number of trials to criterion needed to perform the task (Fig. 3).

Figure 2.

Figure 2.

Open in a new tab

Setup of the experiment consisting of four different tasks involving first order associative learning, latent inhibition, extinction, and reversal learning. For first order associative learning, latent inhibition, and reversal learning, the criterion consisted of 10 consecutive correct trials. For the extinction task, the criterion was met when 10 random errors were committed. Simulated lesions of the MS/VDB were conducted in some of the trials as denoted by the vertical arrow.

Figure 3.

Figure 3.

Open in a new tab

First order associative learning task. Bars represent means ± SD for 100 task runs. Lesion of the MS/VDB did not impair associative learning. Changing the type of the projections (INC) from the Decremental area to the Modulated input area did not have any effect either.

In this task, a lesion of the MS/VDB did not impair associative learning significantly (P-value >0.45). In order to test if an incremental pathway could play the same role as the decremental pathway, we changed the type of the projections between the Decremental area and the Modulated input area from inhibitory to excitatory (INC) and saw that it did not have any significant effect (P-value >0.85). This experiment demonstrated that the MS/VDB was not necessary to perform a simple first order associative learning task.

Neural analysis

Figure 4 shows plots of the neural activity and the strength of plastic synapses changing over time, as well as the number of correct actions performed in a row during a typical run, until the criterion was reached. In this example, the model learned to complete the task in 13 trials (260 time-steps). The first plot at the top of the figure shows the activity of the Input area over time. During the observation period of each trial, both neurons were activated. During the action period, only the neuron corresponding to the selected cue was activated. The second plot shows the activity of the Decremental area. In the third plot, the synaptic strength (weight) of the connections between the Decremental area and the MS/VDB is shown. The model performed this task quite rapidly so the increase of these weights was barely noticeable. In the fourth plot, the MS/VDB did not have the time to learn because the number of trials was small, and thus its activity was minimal. The fifth plot shows the activity of the Modulated input area, which was high since it did not receive a strong inhibitory input from the Decremental area. In the sixth plot, the synaptic strength of a connection between the Modulated input area and the Action selection area is shown. The synaptic strength started to increase rapidly after 30 time-steps due to Hebbian learning modulated by the reward received whenever the model was making a correct action. As the weight kept increasing, the activity of the neuron of the Action selection area corresponding to the correct action kept increasing as well. The last plot shows that the model learned to make the correct choice, and reached the criterion very rapidly.

Figure 4.

Figure 4.

Open in a new tab

Response of the model during first order associative learning task. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the criterion was reached. In this example, the network learned to complete the task in 13 trials (260 time-steps). The first plot at the top of the figure shows the activity of the Input area over time. During the observation period of each trial, both neurons were activated. During the action period, only the neuron corresponding to the selected cue was activated. The second plot shows the activity of the Decremental area. In the third plot, the synaptic strength (weight) of the connections between the Decremental area and the MS/VDB is shown. The model performed this task quite rapidly so the increase of these weights was barely noticeable. In the fourth plot, the MS/VDB did not have the time to learn because the number of trials was small, and thus its activity was minimal. The fifth plot shows the activity of the Modulated input area, which was high since it did not receive a strong inhibitory input from the Decremental area. In the sixth plot, the synaptic strength of a connection between the Modulated input area and the Action selection area is shown. The synaptic strength started to increase rapidly after 30 time-steps due to Hebbian learning modulated by the reward received whenever the model was making a correct action. As the weight kept increasing, the activity of the neuron of the Action selection area corresponding to the correct action kept increasing as well. The last plot shows that the model learned to make the correct choice, and reached the criterion very rapidly. In every run of associative learning task, the MS/VDB did not have sufficient time to learn and its activity was always minimal.

Latent inhibition

Behavioral data

In the latent inhibition task (see Fig. 2, Task 2), we tested the model's ability to condition to a previously exposed stimulus (Chiba et al. 1995; Baxter et al. 1997). During this task, the model had to learn that a previously irrelevant cue predicted a reward. The setup of this experiment was the same as that of the first order associative learning experiments except that it was divided in two phases. During the first phase, no reward was given, causing the model to make random actions. During the second phase, a reward was introduced and the model had to learn that this previously irrelevant cue now predicted a reward. We recorded the number of trials to criterion (Fig. 5) needed to perform the task after the preexposure period of 40 trials (800 time-steps). Figure 5 shows that the intact model (control) exhibited latent inhibition, meaning that more trials were necessary in order to reach the criterion in this experiment than in the associative learning task described above. The number of trials to criterion decreased significantly when the MS/VDB was lesioned (P-value <7.9 × 10−29). Similar to a MS/VDB lesion, changing the type of the projections (INC) from the Decremental area to the Modulated input area also impaired significantly latent inhibition (P-value <1.4 × 10−28). This experiment demonstrated that the cholinergic decremental pathway was necessary for latent inhibition. Lesioning the MS/VDB disrupted latent inhibition and the model performed associative learning as rapidly as in the previous task.

Figure 5.

Figure 5.

Open in a new tab

Latent inhibition task. Bars represent means ± SD for 100 task runs. The intact model exhibited latent inhibition. Lesion of the MS/VDB disrupted latent inhibition and the model performed associative learning very rapidly. Changing the type of the projections (INC) from the Decremental area to the Modulated input area also impaired latent inhibition. (*) Statistically significant difference with P-value <1.4 × 10−28.

Neural analysis

Figure 6 shows the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run with the intact (Control) model. In this example, the model learned to complete the task in 29 trials (580 time-steps) after the preexposure period of 40 trials (800 time-steps). The activity of Neuron 1 in the Decremental area increased rapidly after 820 time-steps, and Neuron 2's activity increased rapidly after 1000 time-steps. The ACh emitted from the MS/VDB caused this change of activity by modulating the gain of targeted neurons. The activity of the MS/VDB increased when the strength of its input synapses also increased. With high activity, the Decremental area inhibited strongly the Modulated input area, causing a large drop in activity. Compared to the previous experiment, the synaptic strength of one of the connections between the Modulated input area and the Action selection area increased quite slowly. As the weight kept increasing, the activity of the neuron of the Action selection area corresponding to the correct action kept increasing slowly as well. These results show that the ACh decremental pathway could indirectly hinder Hebbian learning by regulating the Decremental area to cause a strong inhibition of the Modulated input area, and therefore played an indirect but important role in reinforcement learning. The last plot shows that the model made a certain number of errors before it reached the criterion. This experiment showed that the strong inhibition of the modulated input coming from the decremental area caused a difficulty in learning (latent inhibition).

Figure 6.

Figure 6.

Open in a new tab

Response of the intact model during latent inhibition task. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run until the criterion was reached. In this example, the network learned to complete the task in 29 trials (time-steps 800–1380). The activity of the action selection neuron corresponding to the correct action increased slowly over time due to Hebbian learning modulated by reward. In this example, connections from the Decremental area to the MS/VDB increased slowly over time causing a strong increase of activity after 800–1000 time-steps. ACh from the MS/VDB changed the gain of the Decremental area causing a net increase of activity. The Decremental area then strongly inhibited the Modulated input area causing a difficulty in learning (e.g., latent inhibition).

Figure 7 shows the effect of MS/VDB lesion during this task. In this example, the model learned to complete the task in 12 trials (240 time-steps) after the preexposure period of 40 trials (800 time-steps). The activity of the action selection neuron corresponding to the correct action increased rapidly over time, as in Task 1. In this example, connections from the Decremental area to the MS/VDB increased slowly over time until the MS/VDB was lesioned at 800 time-steps. The Decremental area was not strongly inhibiting the modulated input area, and therefore did not cause a difficulty in learning (i.e., no latent inhibition).

Figure 7.

Figure 7.

Open in a new tab

Response of the model during latent inhibition task with simulated lesion of MS/VDB. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the criterion was reached. In this example, the network learned to complete the task in 12 trials (240 time-steps). The activity of the action selection neuron corresponding to the correct action increased rapidly over time due to Hebbian learning modulated by reward. In this example, connections from the Decremental area to the MS/VDB increased slowly over time until the MS/VDB was lesioned at 800 time-steps. Therefore, the Decremental area was not strongly inhibiting the Modulated input area, and did not cause a difficulty in learning (i.e., latent inhibition).

Extinction

Behavioral data

In the extinction task (see Fig. 2, Task 3), we measured the perseverative behavior of the model toward a previously conditioned stimulus. After learning that a cue predicted a reward, the model had to learn that this previously relevant cue no longer predicted a reward. The setup of this experiment was similar to the latent inhibition experiment except that the two phases were reversed. During the first phase, the model learned that one cue predicted a reward. During the second phase, no reward was given and the model had to stop persisting by not attending to the previously rewarding cue. We recorded the number of trials to extinction criterion needed to perform the task, after the first learning criterion was reached (Fig. 8).

Figure 8.

Figure 8.

Open in a new tab

Extinction task. Bars represent means ± SD for 100 task runs. Lesion of the MS/VDB during the extinction caused a significant increase in perseverative behavior. Changing the type of the projections (INC) from the Decremental area to the Modulated input area also caused stronger perseverative behavior. (*) Statistically significant difference with P-value <1.2 × 10−31.

A simulated lesion of the MS/VDB during the extinction caused a significant increase of perseverative behavior, as indicated by the significantly higher number of trials to extinction criterion needed to perform the task (P-value <7.5 × 10−247). Similar to a MS/VDB lesion, changing the type of the projections (INC) from the decremental area to the modulated input area increased significantly the number of trials to extinction criterion compared to the control group (P-value <5.8 × 10−248) and the lesioned group (P-value <1.2 × 10−31). This experiment demonstrated that the ACh decremental pathway was necessary to decrease attention and stop perseverative behavior during an extinction task.

Neural analysis

Figure 9 shows the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the extinction criterion was reached. In this example, the model reached the first criterion in 10 trials (200 time-steps), and then persisted for 33 more trials (660 time-steps) before reaching the extinction criterion. The activity of the action selection neuron corresponding to the correct action increased rapidly over time until the first criterion was reached. After 640 time-steps, the activity of the decremental area increased rapidly and caused a strong inhibition of the modulated input area. This allowed the model to stop persisting and reach the extinction criterion relatively rapidly.

Figure 9.

Figure 9.

Open in a new tab

Response of the intact model during the extinction task. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the extinction criterion was reached. In this example, the network learned to complete the task in 10 trials (200 time-steps) and then persisted for 33 trials (660 time-steps). The activity of the action selection neuron corresponding to the correct action increased rapidly over time due to Hebbian learning modulated by reward. After 640 time-steps, the activity of the Decremental area increased radically and caused a strong inhibition of the Modulated input area allowing the system to stop persisting.

Figure 10 shows the effect of MS/VDB lesion during this task. In this example, the model learned to complete the task in 11 trials (220 time-steps) but then persisted for 83 more trials (1660 time-steps). The activity of the action selection neuron corresponding to the correct action increased rapidly over time until the first criterion was reached. Due to the lesioned MS/VDB, the decremental area could not strongly inhibit the Modulated input area. The model persisted in choosing the previously rewarded cue for a long time until the synaptic strength of the Action selection area's input connection decayed enough to a point where the resulting activity of the Action selection area was below 0.5 and the extinction criterion was reached.

Figure 10.

Figure 10.

Open in a new tab

Response of the model during an extinction task with a simulated lesion to MS/VDB. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the extinction criterion was reached. In this example, the network learned to complete the task in 11 trials (220 time-steps) but then persisted for 83 trials (1660 time-steps). The activity of the action selection neuron corresponding to the correct action increased rapidly over time due to Hebbian learning modulated by reward. Because of the MS/VDB lesion, the Decremental area could not inhibit strongly the Modulated input area. The system then persisted for a long time until the synaptic strength decreased enough to a point where the resulting activity of the action selection area decreased below 0.5 and the extinction criterion was reached.

Reversal learning

Behavioral data

In the reversal learning task, the model initially had to learn that one cue was predictive of reward and, once the learning criterion was reached, a reversal was introduced and the model had to learn that the other cue was the reward predictor (see Fig. 2, Task 4). We recorded the number of trials needed to reach the criterion during the first period of associative learning, as well the number of trials needed to reach the criterion during the reversal learning period (Fig. 11). We also recorded the number of perseverative and random errors committed by the model (Fig. 12) after the reversal.

Figure 11.

Figure 11.

Open in a new tab

Reversal learning task. Bars represent means ± SD for 100 task runs. Lesion of the MS/VDB during the reversal drastically impaired performance. Changing the type of the projections (INC) from the Decremental area to the Modulated input area also strongly impaired performance. (*) Statistically significant difference with P-value <9.5 × 10−13.

Figure 12.

Figure 12.

Open in a new tab

Reversal learning—error types made after the reversal. A lesion of the MS/VDB greatly increased the number of perseverative errors, showing that the lesioned model took longer to decrease attention to the previous rewarding cue. Similar results were obtained when the type of the projections from the Decremental area to the Modulated input area was changed to excitatory (INC). (*) Statistically significant difference with P-value <1.5 × 10−24.

The intact model performed the task successfully and reached the first criterion quite rapidly but, as expected, had more difficulties after the reversal was introduced and reached the second criterion with a significantly higher number of trials (P-value <2.7 × 10−143). A simulated lesion of the MS/VDB during reversal learning drastically impaired performance and the number of trials to criterion during the reversal period increased significantly compared to the control group (P-value <3.8 × 10−187). Furthermore, the number of perseverative errors committed by this group (Fig. 12) was significantly higher than the number of perseverative errors committed by the control group (P-value <4.9 × 10−234), showing that the lesioned model did not decrease its attention to the previously rewarding cue. Changing the type of the projections (INC) from the decremental area to the modulated input area also strongly impaired performance and the number of trials to criterion increased significantly compared to the control group (P-value <3.3 × 10−193) and the lesioned group (P-value <9.5 × 10−13). The number of perseverative errors committed by the INC group (Fig. 12) was significantly higher than the number of perseverative errors committed by the control group (P-value = 7 × 10−242) and the lesioned group (P-value <1.5 × 10−24). This experiment demonstrated again that the ACh decremental pathway was necessary to decrease attention to a cue that previously predicted a reward, and stop perseverative behavior.

Neural analysis

Figure 13 shows the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the criterion was reached after the reversal. In this example, the model learned the first association in 12 trials (240 time-steps), and the reversal in 40 more trials (800 time-steps). After 700 time-steps, the activity of the decremental area increased rapidly and caused a strong inhibition of the modulated input area allowing the system to stop persisting. The model then learned to associate the other cue with the reward. When the MS/VDB was lesioned after the reversal (Fig. 14), the Decremental area could not inhibit strongly the Modulated input area. The system then persisted for a long time until the synaptic strength of the Action selection area's input connection decreased enough to a point where the resulting activity of the Action selection area was below 0.5. Thereupon, the model made random choices and eventually learned that the other cue was now the reward predictor. In this example, the model learned the first association in 14 trials (280 time-steps), and the reversal in 80 more trials (1600 time-steps). In this experiment, we clearly saw that lesioning the MS/VDB drastically increased the perseverative behavior of the model.

Figure 13.

Figure 13.

Open in a new tab

Response of intact model during the reversal learning task. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the criterion was reached. In this example, the network learned the first association in 12 trials (240 time-steps), and the reversal in 40 trials (800 time-steps).

Figure 14.

Figure 14.

Open in a new tab

Response of the model during a reversal learning with a simulated lesion of the MS/VDB. Plots showing the neural activity, the strength of plastic synapses, and the number of correct actions performed in a row during a typical run, until the criterion was reached. In this example, the network learned the first association in 14 trials (280 time-steps), and the reversal in 80 trials (1600 time-steps). Lesioning the MS/VDB drastically increased the perseverative behavior of the network.

Discussion

The present paper demonstrated that a neuronal pathway with the ability to learn what stimuli should be ignored is important to: (1) decrease attention to a cue that previously predicted a reward; (2) prevent perseverative behavior when reward contingencies change (e.g., in extinction or reversal learning tasks); and (3) show latent inhibition to previously uninteresting cues. However, this pathway was not necessary to increase attention to cues in a task such as first order associative learning. We suggest that these functions are carried out in the brain by cholinergic projections from the medial septum/ventral diagonal band (MS/VDB) to the hippocampus and cingulate. This pathway modulates activity in brain areas to properly allocate the attention to stimuli in the environment necessary for adequate learning to occur and fluid behavior to be maintained. Specifically, our model suggests a mechanism in which the ACh decremental pathway regulates attention through MS/VDB cholinergic projections to areas that can inhibit reinforcement learning. In our model, the ACh decremental pathway hindered Hebbian learning by regulating the Decremental area to strongly inhibit the Modulated input area. If this pathway was altered in our model, the learning necessary for latent inhibition, extinction, and reversal learning was impaired.

Both classic and recent experiments examining attention in rodents have revealed a complex set of processes that act in the service of learning and goal directed action (Holland and Maddux 2010). These may include regulation of overall level of arousal, regulation of the breadth of attention, modulation of attention by temporal factors, direction of action, and, separately, incrementing and decrementing attention to particular variables present in the environment (e.g., to sights or sounds). Classic associative learning theory states that alterations in attentional processing, defined as decremental, occur when a stimulus provides no new predictive information, whereas alterations in attentional processing in the form of incremental changes occur when expectations regarding future events are violated (Pearce and Hall 1980). Based on this notion, augmentation of attention to the stimulus should be achieved by altering its relationship with future events. A great deal of behavioral data gathered from intact rats supports this claim (Pearce et al. 1988; Wilson et al. 1992; Holland and Gallagher 1993). A body of work demonstrated that the basal forebrain cholinergic projections emanating from the SI/nBM to the parietal cortex are essential for incrementing attention (Chiba et al. 1995; Bucci et al. 1998), whereas the basal forebrain cholinergic projections emanating from the MS/VDB region of the basal forebrain are essential for decrementing attention (Han et al. 1995; Baxter et al. 1997). Thus, a double dissociation between the subset of cholinergic projections essential for incremental and decremental attention is evident. Despite the discrete projections of these neuronal subregions, they exist as a continuum within a large swath of the basal forebrain, with adequate opportunity to interact at the level of the basal forebrain and again at the level of the prefrontal cortex (Zaborszky 2002). Reflective of this anatomy, these subregions must act both independently and synchronously in situations that go beyond the elegance and control of classical conditioning tasks. Learning environments that possess the noisy statistics of the natural ecology mandate a synthesis of incremental and decremental processes, enabling fluid attentional performance and efficient learning.

Cholinergic modulation is thought to increase attention and encode novel information. It has been proposed that cholinergic neurons mediate selective increases in attention in proportion to the probabilistic uncertainty of stimulus predictions, a degree of modulation that would facilitate stimulus detection, and would theoretically mediate selective increases of attention to stimuli that can predict relevant outcomes in the environment (Yu and Dayan 2002). However, such theories do not accommodate for decremental processes that are undoubtedly intermingled and essential for balancing expectations about the world. Hasselmo also proposed, from experimental evidence, that acetylcholine may enhance encoding by augmenting the influence of external afferent input, and reduce the synaptic transmission at excitatory recurrent feedback synapses within cortical structures (Hasselmo and McGaughy 2004). While these processes could enhance external afferent input and filter out irrelevant information, they do not involve a separate cholinergic pathway that learns to ignore.

In order to test hypotheses concerning decrementing attention, we focused on the way a specific pathway can decrease attention by regulating reinforcement learning and action selection. We introduced simple neural mechanisms involving ACh gain modulation and inhibition as a way to regulate conditioned stimulus processing, reinforcement learning, and action selection. The model presented in this work was able to replicate experimental results of different studies where rats had to perform similar tasks, and was used to make a prediction on the role the MS/VDB on reversal learning (Table 1).

Table 1.

Comparison of the behavior between the simulated MS/VDB lesioned model and similar experimental studies

graphic file with name OROSLM032433TB1.jpg

Open in a new tab

Role of the MS/VDB during first order associative learning

We observed that the MS/VDB in our model played no significant role during first order associative learning (Table 1; Figs. 3, 4). The model performed this task quite rapidly and the MS/VDB did not have the time to learn which cue was irrelevant, and thus its activity and ACh contribution were minimal. In this simple task where the cues were novel, the model did not have time to learn a priori so the MS/VDB played no significant role during first order associative learning.

A lesion of the cholinergic MS/VDB neurons in rats also showed no effect on learning a rewarded location in a single environment (Table 1; Janisiewicz et al. 2004). In this study, rats that received lesions of cholinergic neurons in the MS/VDB were tested on a conditional associative learning task. The authors constructed two environments using an octagon and a cube, which displayed different visual cues on their walls. The location of reward differed between the two environments. Rats were trained on one environment and then introduced to a second environment for discrimination, or rats were trained on both environments concurrently. The findings of these experiments showed that MS/VDB lesions had no effect on learning a rewarded location in a single environment. However, MS/VDB lesioned rats were impaired when the two parts (environments) of the conditional problem were presented concurrently, suggesting that active suppression of attention was necessary in this case. Janisiewicz and Baxter (2003) looked at the performance of rats on a conditional task, in which one or two visual stimuli directed the rat to respond to an illuminated panel on the left or right. Control rats with experience on other visual tasks performed better than MS/VDB-lesioned rats with similar experience, whereas this effect was reversed in naive rats. From these results, the authors suggested that MS/VDB cholinergic neurons play a particular role in the transfer of behavioral experience, rather than a specific role in associative learning.

Role of the MS/VDB during latent inhibition

In our experiments, simulated lesions of the MS/VDB showed that it was necessary for latent inhibition, a difficulty in conditioning to a previously exposed stimulus (Table 1; Figs. 5–7). The intact model exhibited latent inhibition, meaning that more trials were necessary in order to reach the criterion in this experiment than in the previous one. The latent inhibition simulations suggest that the ACh decremental pathway could hinder reinforcement learning indirectly, by causing the Decremental area to cause a strong inhibition of the Modulated input area. The number of trials to criterion decreased significantly when the MS/VDB was lesioned. Therefore, lesioning the MS/VDB disrupted latent inhibition and the model performed associative learning as rapidly as in the previous task. In the lesion case, the Decremental area was not modulated by ACh, did not strongly inhibit the Modulated input area, and thus did not interfere with learning (i.e., no latent inhibition). From these and the previous experiments, we predict that injections of an ACh agonist into the target of the MS/VDB (e.g., cingulate, hippocampus) during early task learning would cause a difficulty in reinforcement learning similar to latent inhibition. That is, attention to the new stimuli would be suppressed.

Our results are comparable to experimental evidence showing that lesions of cholinergic projections from the MS/VDB to the hippocampus disrupt latent inhibition (Table 1; Baxter et al. 1997). In this study, an experiment was designed to test latent inhibition, which is the slower learning of a previously exposed, uninformative stimulus. Control rats showed diminished conditioning to a preexposed stimulus, as compared with the novel stimulus. However, conditioning to the novel and preexposed cues occurred at the same rate in the MS/VDB lesioned rats.

Role of the MS/VDB during extinction

A lesion of the MS/VDB in our model disrupted extinction by increasing perseverative behavior (Table 1; Figs. 8–10), as indicated by the significantly higher number of trials to an extinction criterion needed to perform the task. This experiment demonstrated that the ACh decremental pathway was necessary to decrease attention and prevent perseverative behavior.

Similarly, experiments showed that rats with nonspecific lesions of the MS had an increased resistance to extinction (Table 1; Feldon and Gray 1979). In this study, experiments were conducted in a straight alley with food reward on continuous or partial reinforcement. The authors found that MS lesions greatly increased resistance to extinction especially during partial reinforcement. However, a lesion of the dorsolateral septal (LS) area had an inverse effect as a decrease in resistance to extinction was observed. Their results demonstrated a clear double dissociation between the effects of MS and LS lesions, especially for the partial reinforcement condition. We predict that specific ACh lesions of the MS/VDB would show a similar effect where rats would exhibit strong perseverative behavior.

Role of the MS/VDB during reversal learning

During reversal learning, the number of trials to criterion increased significantly when the MS/VDB was lesioned, and resulted in a significant increase in the number of perseverative errors (Table 1; Figs. 11–14). In this experiment, we showed that our model could generalize to another form of learning using the same mechanisms where we demonstrated that the ACh decremental pathway was important to decrease attention and stop perseverative behavior. To the best of our knowledge, no experimental evidence currently exists to support or disprove this claim. This leads us to predict that the cholinergic lesions of the MS/VDB in rats would impair reversal learning and cause stronger perseverative behavior.

Many studies have shown different effects of brain lesions during reversal learning (Ragozzino and Choi 2004; Floresco et al. 2008; Clarke et al. 2011). Most relevant to the present work, Tait and Brown (2008) showed that selective lesions of the SI/nBM cholinergic neurons did not result in impairments of reversal learning or set-shifting. However nonselective lesions impaired reversal learning and interfered with the formation of an attentional set. Moreover, inactivation of the dorsal anterior cingulate in rats was shown to increase never reinforced errors, but not perseveration (Ragozzino and Rozman 2007). Furthermore, lesions of the medial prefrontal cortex and hippocampus increased perseverative errors. By contrast, lesions of the anterior cingulate cortex increased random errors (Kosaki and Watanabe 2012). Our model focused on the MS/VDB and its possible influence on many of these areas, such as the hippocampus and the cingulate, including the anterior cingulate, which is part of the mPFC. We would predict that a cholinergic antagonist in these areas would impair reversal learning.

Neuroanatomy of the decremental pathway

We have to emphasize that our abstract model was used to implement attentional processes that may not take the form of distinct areas, but instead suggests canonical operations occurring within executive, sensory, and motor systems. However, an analogy between our model and known neural circuits can be attempted.

Baxter hypothesized (Baxter et al. 1997) that the hippocampus projected to cholinergic neurons in the MS/VDB in order to regulate processing in cortical (cingulate) and subcortical targets (hippocampus). Evidence for projections from the MS/VDB to the cingulate (Gaykema et al. 1990; Hoover and Vertes 2007), and from the hippocampus to the MS/VDB (Swanson 1977; Gaykema et al. 1991) supports this theory. Alternatively, the hippocampus projects indirectly to the basal forebrain through the nucleus accumbens (NAc) (Zahm 1999). Weiner's switching model of latent inhibition also involves hippocampus projections to the NAc (Weiner and Feldon 1997).

In our model, the Decremental area in Figure 1 roughly corresponds to the hippocampus and cingulate combined. The Decremental area projects to cholinergic neurons in the MS/VDB. These neurons project back to the Decremental area. Finally, the Decremental area projects to an Action selection area with inhibitory connections. Based on our model and experiments, we predict that ACh from the MS/VDB modulates the outputs of the hippocampus and cingulate having inhibitory effects on an action selection area, possibly the basal ganglia.

In our model cholinergic projections from the MS/VDB have specific neural targets in the Decremental area. Although such a direct mapping has not been shown experimentally, contemporary anatomical theories suggest a highly differentiated topographic organization of the projections from the cholinergic system (Price and Stern 1983; Walker et al. 1985; Koliatsos et al. 1988; Gaykema et al. 1990; Zaborszky 2002; Zaborszky et al. 2005, 2008). Furthermore, the debate on targeted versus volume transmission of ACh is still ongoing, and evidence has been shown to support both modes (Sarter et al. 2009b). Finally, the targeted effect of ACh might also be explained by glutamatergic projections from the thalamic mediodorsal (MD) nucleus projections to the PFC, which target the terminals of cholinergic neurons that mediate attentional processes (Sarter et al. 2009a; Hasselmo and Sarter 2011; Sarter and Paolone 2011).

Contribution of present model

In the present paper, we constructed and formalized a neural network, which was based on a theoretical model and experimental evidence for decremental attention (Baxter et al. 1997), to replicate experimental results. From this simulation work, we make the following predictions: (1) The ACh decremental pathway helps prevent perseverative behavior and this would be evident if this pathway was lesioned during a reversal learning task; (2) injections of an ACh agonist into the targets of the MS/VDB (e.g., cingulate, hippocampus) during early task learning would cause a difficulty in reinforcement learning; (3) specific ACh lesions in the MS/VDB would increase perseverative behavior during an extinction task; and (4) ACh from the MS/VDB modulates the outputs of the hippocampus and cingulate cortex that can result in inhibitory effects on an action selection area (e.g., the basal ganglia).

To the best of our knowledge, no modeling studies have investigated how ACh can decrement attention using a specific pathway and showed its role in tasks involving associative learning, latent inhibition, extinction, and reversal learning. Our goal was to use a relatively simple model in order to investigate the roles of the cholinergic decremental pathway in a model performing these tasks. In this paper, we demonstrated how acetylcholine from the MS/VDB can modify the neural dynamics within a decremental attentional pathway, and also demonstrated how ACh from the MS/VDB was important to decrease attention to nonrewarding cues. Changing the Decremental area into an Incremental area (INC) resulted in the inability to demonstrate latent inhibition and reduce perseverative behavior, emphasizing the role of the ACh decremental pathway. We implemented acetylcholine in our model based on biological mechanisms where ACh from the MS/VDB increased the gain of input connections from sensory neurons to neurons in the Decremental area. Experimental evidence has shown that acetylcholine modulates cortical pyramidal cells in order to enhance the response to sensory input, therefore changing the gain of input connections to target neurons (Hasselmo and McGaughy 2004; Sarter et al. 2005).

The role of ACh in attention has been investigated with several computational models, but these models have focused mainly on incrementing attention and the SI/nBM portion of the basal forebrain (Grossberg and Versace 2008; Deco and Thiele 2011; Avery et al. 2012a). For example, Yu and Dayan (2005) developed a Bayesian model where cholinergic and noradrenergic (NE) systems responded to expected and unexpected uncertainty, respectively, in an extended Posner task. For each trial, they presented five different colored cues pointing right or left, then a target was shown at one location (left or right). One of the cues predicted the target location with a certain probability. ACh in their model tracked expected uncertainty that corresponded to the invalidity of the predictive cue. NE tracked unexpected uncertainty that was introduced in the task by changing the cue predicting the target location (e.g., from red to blue). Lesions of these neuromodulatory systems decreased the performance of their model, which led to increased perseverative errors. Following Yu and Dayan's ideas, Avery et al. (2012b) created a neural model of the cholinergic and noradrenergic systems tracking uncertainty and, in turn, influencing cortical processing through nicotinic enhancement of thalamocortical input, muscarinic regulation of corticocortical feedback, noradrenergic mediation of a network reset, locus coeruleus (LC) activation of the basal forebrain (BF), and cholinergic and noradrenergic balance between sensory input and frontal cortex predictions. This model focused on the influence of ACh and NE neuromodulation on a bottom-up versus a top-down pathway. In this model, lesions of the neuromodulatory systems also decreased performance and resulted in more perseverative errors. Both Yu's and Avery's models captured the incremental aspect of attention but did not accommodate for decremental attentional processes. Although these models implicate the noradrenergic system in preventing perseverative behavior, we doubt that they would exhibit latent inhibition as shown by our model and empirical evidence, which implicates the ACh decremental pathway as necessary for latent inhibition. In the future, models used to study the role of ACh on attention should not only focus on the incremental pathway, but should also include the decremental pathway.

Previous models have demonstrated latent inhibition (Schmajuk et al. 1996; Weiner and Feldon 1997). Most pertinent to our work, Gomond and Salotti (2006) built a neural network that could perform associative learning and also exhibit latent inhibition. This model was based on the Schmajuk–Lam–Gray (SLG) neural network model (Schmajuk 2005) that can exhibit many of the properties of latent inhibition. In the SLG model, predictions of future events are calculated and the system compares observed and predicted events in order to compute novelty. Novelty is then used to control attention to stimuli, modify predictions, and inhibit ongoing behavior. Gomond's model was composed of three distinct modules, one for the perception of stimuli, one for associative learning, and one for the selection of behaviors. The model included “time battery” units that acted as delay neurons and were mainly responsible for conditioning, prediction making, and the associative learning between perception, motivation, and action. Their architecture was designed as an action-selection mechanism for the processing of the stimuli and also for the perception–reward–action learning. However, their model did not include cholinergic modulation and study its role in latent inhibition. Moreover, their model was not based on neurophysiological findings and therefore did not make any predictions on specific mechanisms involved in the brain.

We should emphasize that the study of ACh levels in the hippocampus, although seemingly important for encoding information (Hasselmo and McGaughy 2004; Hasselmo 2006; Micheau and Marighetto 2011), was not the focus of the research presented here. In our model, the level of ACh is initially low, and increases slowly over time as the MS/VDB learns. This response profile was used to model the complex interactions between the hippocampus, the MS/VDB, and the cingulate, where the decremental pathway has been shown to be necessary for latent inhibition. However, we believe that the ACh release from MS/VDB neurons in a real brain would show complex patterns with phasic and tonic components. Although phasic cholinergic transients in the PFC have been shown to be important for cue detection (Parikh et al. 2007; Parikh and Sarter 2008), a tonic level of ACh has also been observed during entire sessions which might be correlated with performance (Parikh et al. 2007; Parikh and Sarter 2008). Furthermore, we did not consider the MS/VDB's role in encoding novel memories as has been shown by others. However, based on our experiments, we would predict that injections of an ACh agonist into the target of the decremental pathway (e.g., cingulate, hippocampus) during early task learning would cause a difficulty in learning new memories because it would suppress attention to the novel stimuli, in a similar fashion to latent inhibition. It would be of interest to test the present model of decrementing attention in a memory task similar to those of Hasselmo and McGaughy (2004), Hasselmo (2006), and Micheau and Marighetto (2011).

Finally, the ability to increase attention to novel or uncertain stimuli requires the ACh incremental pathway and not the decremental pathway (see above). An interesting extension to this work would be to model both pathways and show how they could be complementary and cover different aspects of attentional processes such as increasing responses to salient, unexpected, and rewarding stimuli, as well as ignoring irrelevant ones, but also disregard salient and novel stimuli if needed.

In summary, we developed a neural simulation to provide insight into how acetylcholine can decrement attention using a specific pathway. We simulated a behavioral paradigm to probe decremental attention in the context of reinforcement learning. Simulations showed the ACh decremental pathway, which we suggest is from the MS/VDB to the cingulate and hippocampus, was necessary for latent inhibition, a difficulty in conditioning to a previously exposed stimulus, and crucial for decreasing attention to an irrelevant stimulus that was initially predicting a reward and stopping perseverative behavior. In addition, the model predicts an important role for the cholinergic decremental pathway during a reversal learning task in order to decrease perseverative behavior. Taken together, the model demonstrates that the ACh decremental pathway is necessary for appropriate learning and attention under dynamic circumstances and suggests a canonical neural architecture for decrementing attention.

Materials and Methods

Neural architecture

A neural network abstracting the known functional neuroanatomy for attentional pathways was constructed (Fig. 1A). It consisted of an Input area, a Decremental area (which loosely corresponds to the hippocampus and cingulate), a Modulated input area, an Action selection area and one neuromodulatory cholinergic area (MS/VDB). Every neural area had two neurons corresponding to each cue. Between different neural areas, these neurons were connected through “one-to-one” synaptic projections. Neurons of the Input area fired when the model was presented with one or both cues. The Input area provides information on the cues presented to the rest of the model by projecting to the Decremental area and the Modulated input area. The Decremental area was connected to the MS/VDB via plastic synapses, and connected to the Modulated input with inhibitory synapses. Acetylcholine originating from the MS/VDB modified the gain of neurons in the Decremental area. The role of the MS/VDB was to learn when cues were exposed for a long time and increase the activity of the Decremental area specific to those cues via cholinergic neuromodulation. The role of the Decremental area was to decrement attention by strongly inhibiting neurons in the Modulated input area. Otherwise, it caused only moderate inhibition. The Modulated input area projected to the Action selection area via plastic connections that were reinforced by reward. The role of this subnetwork was to learn to associate a cue with a reward, and select the appropriate action leading to this reward.

Model of neurons

We used a mean firing rate neuron model defined by:

graphic file with name OrosLM032433eq01.jpg (1)

where t is the current time-step, Si is the activation level of neuron i, Gi is the neuronal gain, Ii is the total synaptic input, and θi is a bias that shifts the range of sensitivity of the sigmoid function. One time-step in our simulation corresponded to ∼100 ms of real time. We set Gi = 8 and θi = 0.6 for the Modulated input area, and Gi = 5 and θi = 0.3 for the Action selection area. For all other neurons, the value of Gi was set to 10 and θi to 0.5. These parameter settings were chosen in order to cover the full range of the sigmoid curve based on the synaptic input into a neuron.

The synaptic input of a neuron was based on presynaptic neural activity, and the connection strength of the synapse:

graphic file with name OrosLM032433eq02.jpg (2)

where wij is the synaptic weight from neuron j to neuron i, Inoise is additive noise with a small random value drawn from a uniform distribution (−0.025; 0.025). The default value of the weights for all nonplastic projections was set to +1 for excitatory and −1 for inhibitory connections. The Modulated input area received a strong input from the Input area (w = 3). The initial value of all plastic weights was set to 0.1.

Neuromodulation

Connections from the Input area to the Decremental area were modulated by acetylcholine from the MS/VDB. For the Decremental area, the synaptic input of a neuron was based on presynaptic neural activity, the connection strength of the synapse, and the amount of neuromodulatory activity:

graphic file with name OrosLM032433eq03.jpg (3)

where AChiMS is the activity of a cholinergic neuron i from the MS/VDB. In our model, ACh increased the gain of input connections from sensory neurons (Input) to neurons in the Decremental area (Hasselmo and McGaughy 2004; Sarter et al. 2005; Disney et al. 2007).

Hebbian learning

In our model, connections from the Decremental area to the MS/VDB were subject to Hebbian learning where the change of synaptic strength was defined by:

graphic file with name OrosLM032433eq04.jpg (4)

where Δwij(t) is the change of weight, ε is the decay rate (0.0001), and δ is the learning rate (0.04). The maximum value of w was capped at one.

Connections from the Modulated input to the Action selection area were subject to Hebbian learning modulated by reward:

graphic file with name OrosLM032433eq05.jpg (5)

where Δwij(t) is the change of weight, ε is the decay rate (0.001), δ is the learning rate (0.1), and R(t) was equal to one if a reward was given, zero otherwise. The maximum value of w was capped at one. A reward was given during the entire action period (Fig. 1B) if the model selected the correct action leading to the rewarding cue (see below).

Action selection

During the observation period of a trial (Fig. 1B), the model was presented with two cues. At the end of the observation period, a cue was selected based on the activity of the neurons in the Action selection area. The neuron with the highest activity above 0.5 selected the cue. If no neuron reached an activity level of 0.5, a cue was randomly selected. The selected cue was then the only one presented to the model during the action period (Fig. 1B). A reward was given during the entire action period if the model selected the correct cue.

Lesions

Simulated lesions of the MS/VDB cholinergic neurons were conducted during each experiment in order to elucidate the role of the decremental pathway by setting the neural activity of the MS/VDB to zero. A simulated lesion would only persist during one phase of a task (Fig. 2). The neural activity during the other phases was computed normally. In order to test if an incremental pathway could play the same role as the decremental pathway, we performed experiments where we changed the type of the projections from the Decremental area to the Modulated input area, from inhibitory to excitatory. Experiments performed with this condition were denoted as INC. Comparing the results with the control experiments would show if an incremental pathway could perform the same function as the decremental pathway or not. For each of the experiments conducted, we performed two-sample t-tests with Bonferroni correction in order to find significant differences between groups (control, lesion MS/VDB, INC).

Data analysis

Each experiment consisted of a series of 100 runs for which we recorded the number of trials to criterion (10 consecutive correct trials). In the case of extinction, the criterion was reached when the model made 10 random errors. An error was considered random when a nonrewarding cue was randomly selected (i.e., when the activity of action selection neurons was <0.5). An error was considered perseverative when the model selected a previously rewarding cue (i.e., when the activity of action selection neurons was ≥0.5). We also recorded the neural activity and synaptic strengths of the model, as well as the number of correct actions performed in a row by the model.

Acknowledgments

Supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of the Interior (DOI) contract number D10PC20021, and NSF award number IIS-0910710. (The US Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DOI, or the US Government.)

References

  1. Avery M, Krichmar JL, Dutt N 2012a. Spiking neuron model of basal forebrain enhancement of visual attention. In The 2012 international joint conference on neural networks (IJCNN), pp. 1–8, Brisbane, Australia [Google Scholar]
  2. Avery MC, Nitz DA, Chiba AA, Krichmar JL 2012. b. Simulation of cholinergic and noradrenergic modulation of behavior in uncertain environments. Front Comput Neurosci 6: 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baxter MG, Holland PC, Gallagher M 1997. Disruption of decrements in conditioned stimulus processing by selective removal of hippocampal cholinergic input. J Neurosci 17: 5230–5236 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bucci DJ, Holland PC, Gallagher M 1998. Removal of cholinergic input to rat posterior parietal cortex disrupts incremental processing of conditioned stimuli. J Neurosci 18: 8038–8046 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chiba AA, Bucci DJ, Holland PC, Gallagher M 1995. Basal forebrain cholinergic lesions disrupt increments but not decrements in conditioned stimulus processing. J Neurosci 15: 7315–7322 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Clarke HF, Hill GJ, Robbins TW, Roberts AC 2011. Dopamine, but not serotonin, regulates reversal learning in the marmoset caudate nucleus. J Neurosci 31: 4290–4297 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Deco G, Thiele A 2011. Cholinergic control of cortical network interactions enables feedback-mediated attentional modulation. Eur J Neurosci 34: 146–157 [DOI] [PubMed] [Google Scholar]
  8. Disney AA, Aoki C, Hawken MJ 2007. Gain modulation by nicotine in macaque v1. Neuron 56: 701–713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Feldon J, Gray JA 1979. Effects of medial and lateral septal lesions on the partial reinforcement extinction effect at one trial a day. Q J Exp Psychol 31(Pt 4): 653–674 [DOI] [PubMed] [Google Scholar]
  10. Floresco SB, Block AE, Tse MT 2008. Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure. Behav Brain Res 190: 85–96 [DOI] [PubMed] [Google Scholar]
  11. Gallagher M, Holland PC 1994. The amygdala complex: Multiple roles in associative learning and attention. Proc Natl Acad Sci 91: 11771–11776 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gaykema RP, Luiten PG, Nyakas C, Traber J 1990. Cortical projection patterns of the medial septum–diagonal band complex. J Comp Neurol 293: 103–124 [DOI] [PubMed] [Google Scholar]
  13. Gaykema RP, van der Kuil J, Hersh LB, Luiten PG 1991. Patterns of direct projections from the hippocampus to the medial septum–diagonal band complex: Anterograde tracing with Phaseolus vulgaris leucoagglutinin combined with immunohistochemistry of choline acetyltransferase. Neuroscience 43: 349–360 [DOI] [PubMed] [Google Scholar]
  14. Gomond N, Salotti JM 2006. Extended model of conditioned learning within latent inhibition. In 14th European symposium on artificial neural networks (ESANN 2006), pp. 89–94, Bruges, Belgium [Google Scholar]
  15. Grossberg S, Versace M 2008. Spikes, synchrony, and attentive learning by laminar thalamocortical circuits. Brain Res 1218: 278–312 [DOI] [PubMed] [Google Scholar]
  16. Han JS, Gallagher M, Holland P 1995. Hippocampal lesions disrupt decrements but not increments in conditioned stimulus processing. J Neurosci 15: 7323–7329 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Han JS, Holland PC, Gallagher M 1999. Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats. Behav Neurosci 113: 143–151 [DOI] [PubMed] [Google Scholar]
  18. Hasselmo ME 2006. The role of acetylcholine in learning and memory. Curr Opin Neurobiol 16: 710–715 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hasselmo ME, McGaughy J 2004. High acetylcholine levels set circuit dynamics for attention and encoding and low acetylcholine levels set dynamics for consolidation. Prog Brain Res 145: 207–231 [DOI] [PubMed] [Google Scholar]
  20. Hasselmo ME, Sarter M 2011. Modes and models of forebrain cholinergic neuromodulation of cognition. Neuropsychopharmacology 36: 52–73 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Holland PC 2007. Disconnection of the amygdala central nucleus and the substantia innominata/nucleus basalis magnocellularis disrupts performance in a sustained attention task. Behav Neurosci 121: 80–89 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Holland PC, Gallagher M 1993. Amygdala central nucleus lesions disrupt increments, but not decrements, in conditioned stimulus processing. Behav Neurosci 107: 246–253 [DOI] [PubMed] [Google Scholar]
  23. Holland P, Maddux J 2010. Brain systems of attention in associative learning. In Attention and learning (ed. Mitchell CJ, LePelley ME), pp. 305–349 Oxford University Press, Oxford, UK [Google Scholar]
  24. Hoover WB, Vertes RP 2007. Anatomical analysis of afferent projections to the medial prefrontal cortex in the rat. Brain Struct Funct 212: 149–179 [DOI] [PubMed] [Google Scholar]
  25. Janisiewicz AM, Baxter MG 2003. Transfer effects and conditional learning in rats with selective lesions of medial septal/diagonal band cholinergic neurons. Behav Neurosci 117: 1342–1352 [DOI] [PubMed] [Google Scholar]
  26. Janisiewicz AM, Jackson O, Firoz EF, Baxter MG 2004. Environment-spatial conditional learning in rats with selective lesions of medial septal cholinergic neurons. Hippocampus 14: 265–273 [DOI] [PubMed] [Google Scholar]
  27. Koliatsos VE, Martin LJ, Walker LC, Richardson RT, DeLong MR, Price DL 1988. Topographic, non-collateralized basal forebrain projections to amygdala, hippocampus, and anterior cingulate cortex in the rhesus monkey. Brain Res 463: 133–139 [DOI] [PubMed] [Google Scholar]
  28. Kosaki Y, Watanabe S 2012. Dissociable roles of the medial prefrontal cortex, the anterior cingulate cortex, and the hippocampus in behavioural flexibility revealed by serial reversal of three-choice discrimination in rats. Behav Brain Res 227: 81–90 [DOI] [PubMed] [Google Scholar]
  29. Lubow RE, Weiner I 2010. Latent inhibition: Cognition, neuroscience, and applications to schizophrenia. Cambridge University Press, Cambridge, UK [Google Scholar]
  30. Micheau J, Marighetto A 2011. Acetylcholine and memory: A long, complex and chaotic but still living relationship. Behav Brain Res 221: 424–429 [DOI] [PubMed] [Google Scholar]
  31. Parikh V, Sarter M 2008. Cholinergic mediation of attention: Contributions of phasic and tonic increases in prefrontal cholinergic activity. Ann N Y Acad Sci 1129: 225–235 [DOI] [PubMed] [Google Scholar]
  32. Parikh V, Kozak R, Martinez V, Sarter M 2007. Prefrontal acetylcholine release controls cue detection on multiple timescales. Neuron 56: 141–154 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Pearce JM, Hall G 1980. A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87: 532–552 [PubMed] [Google Scholar]
  34. Pearce JM, Kaye H, Wilson PN 1988. The influence of predictive accuracy on serial conditioning with rats. Q J Exp Psychol 40B: 181–198 [Google Scholar]
  35. Price JL, Stern R 1983. Individual cells in the nucleus basalis–diagonal band complex have restricted axonal projections to the cerebral cortex in the rat. Brain Res 269: 352–356 [DOI] [PubMed] [Google Scholar]
  36. Ragozzino ME, Choi D 2004. Dynamic changes in acetylcholine output in the medial striatum during place reversal learning. Learn Mem 11: 70–77 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Ragozzino ME, Rozman S 2007. The effect of rat anterior cingulate inactivation on cognitive flexibility. Behav Neurosci 121: 698–706 [DOI] [PubMed] [Google Scholar]
  38. Sarter M, Paolone G 2011. Deficits in attentional control: Cholinergic mechanisms and circuitry-based treatment approaches. Behav Neurosci 125: 825–835 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sarter M, Hasselmo ME, Bruno JP, Givens B 2005. Unraveling the attentional functions of cortical cholinergic inputs: Interactions between signal-driven and cognitive modulation of signal detection. Brain Res Brain Res Rev 48: 98–111 [DOI] [PubMed] [Google Scholar]
  40. Sarter M, Parikh V, Howe WM 2009a. nAChR agonist-induced cognition enhancement: Integration of cognitive and neuronal mechanisms. Biochem Pharmacol 78: 658–667 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sarter M, Parikh V, Howe WM 2009b. Phasic acetylcholine release and the volume transmission hypothesis: Time to move on. Nat Rev Neurosci 10: 383–390 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Schmajuk N 2005. Brain–behaviour relationships in latent inhibition: A computational model. Neurosci Biobehav Rev 29: 1001–1020 [DOI] [PubMed] [Google Scholar]
  43. Schmajuk NA, Gray JA, Lam YW 1996. Latent inhibition: A neural network approach. J Exp Psychol Anim Behav Process 22: 321–349 [DOI] [PubMed] [Google Scholar]
  44. Swanson LW 1977. The anatomical organization of septo-hippocampal projections. Ciba Found Symp 58: 25–48 [DOI] [PubMed] [Google Scholar]
  45. Tait DS, Brown VJ 2008. Lesions of the basal forebrain impair reversal learning but not shifting of attentional set in rats. Behav Brain Res 187: 100–108 [DOI] [PubMed] [Google Scholar]
  46. Walker LC, Kitt CA, DeLong MR, Price DL 1985. Noncollateral projections of basal forebrain neurons to frontal and parietal neocortex in primates. Brain Res Bull 15: 307–314 [DOI] [PubMed] [Google Scholar]
  47. Weiner I, Feldon J 1997. The switching model of latent inhibition: An update of neural substrates. Behav Brain Res 88: 11–25 [DOI] [PubMed] [Google Scholar]
  48. Wilson PN, Boumphrey P, Pearce JM 1992. Restoration of the orienting response to a light by a change in its predictive accuracy. Q J Exp Psychol 44B: 17–36 [Google Scholar]
  49. Yu AJ, Dayan P 2002. Acetylcholine in cortical inference. Neural Netw 15: 719–730 [DOI] [PubMed] [Google Scholar]
  50. Yu AJ, Dayan P 2005. Uncertainty, neuromodulation, and attention. Neuron 46: 681–692 [DOI] [PubMed] [Google Scholar]
  51. Zaborszky L 2002. The modular organization of brain systems. Basal forebrain: The last frontier. Prog Brain Res 136: 359–372 [DOI] [PubMed] [Google Scholar]
  52. Zaborszky L, Buhl DL, Pobalashingham S, Bjaalie JG, Nadasdy Z 2005. Three-dimensional chemoarchitecture of the basal forebrain: Spatially specific association of cholinergic and calcium binding protein-containing neurons. Neuroscience 136: 697–713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Zaborszky L, Hoemke L, Mohlberg H, Schleicher A, Amunts K, Zilles K 2008. Stereotaxic probabilistic maps of the magnocellular cell groups in human basal forebrain. Neuroimage 42: 1127–1141 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Zahm DS 1999. Functional–anatomical implications of the nucleus accumbens core and shell subterritories. Ann N Y Acad Sci 877: 113–128 [DOI] [PubMed] [Google Scholar]

Articles from Learning & Memory are provided here courtesy of Cold Spring Harbor Laboratory Press

RESOURCES