Figure 1.

Hypothetical functional role of neuromodulated synaptic plasticity. (A–C) Reward-modulated learning (A) Schematic reward-based learning experiment. An animal learns to perform a desired sequence of actions (e.g., move straight, then turn left) in a T-maze through trial-and-error with rewards (cheese symbol represents location of reward). (B) The current position (“place”) of the animal in the environment is represented by an assembly of active cells in the hippocampus. These cells feed neurons (e.g., in the dorsal striatum) which code for high-level actions at the choice point, e.g., “turn left” or “turn right.” These neurons in turn project to motor cortex neurons, responsible for the detailed implementation of actions. A success signal, representing the outcome of the actions at the behavioral level (i.e., food or no food), modulates (green arrows) the induction of plasticity at those synapses that have been marked by coincident pre- and postsynaptic activity (solid black connections), but not of those synapses where either the pre- or the postsynaptic neuron was silent (dashed connections). Note that several intermediate layers are possible between each brain area. (C) Neuromodulatory timing. While action potentials of pre- and postsynaptic neurons occur on the time scale of milliseconds, the success signal, representing “reward minus expected reward,” occurs much later (broken time axis). (D–F) Novelty-modulated learning in a neural network. (D) Novelty is defined by the occurrence of a stimulus that does not match pre-existing experience. In this example, a neural network has been trained to recognize an apple. The first time it sees a banana recognition fails and a novelty signal is triggered. (E) Schematic of a neuromodulated neural network for novelty-based learning tasks. Neural assemblies represent known concepts. Here a “banana” stimulus is presented, failing to activate the “apple” neurons, but activating a group of other neurons which will, in the future, encode the “banana” concept. The novelty signal, concurrent with pre- and postsynaptic activation of the banana neurons ensures that synapses (solid lines) between neurons of the banana “assembly” are strengthened. The synapses of the “apple” assembly receive the same neuromodulatory signal, but do not change because pre- or postsynaptic neurons are not simultaneously active. (F) Neuromodulatory timing. Contrary to the reward-based case, the novelty signal can be synchronous with neural activity, or arise slightly earlier or later. (G) Schematic of relation between neuromodulators and functional roles. A specific neuromodulator (e.g., dopamine) could transmit a signal conveying a mixture of novelty and success (green axes). A novel event (gray dot) can be at the same time surprising and rewarding and cause the simultaneous emission of acetylcholine and dopamine, in different proportions (black axes).