Figure 1.

Model of how cortex and basal ganglia lead to action selection during instrumental conditioning. The premotor cortex selects an action (winner-take-all) based on biases for each action received from the basal ganglia. Different potential actions (left/right, L/R) are represented by different groups of medium spiny neurons (MSN) in the striatum in both the direct and indirect pathways. The direct pathway is termed (Go) because activity in this pathway creates a positive bias for an action, whereas the indirect pathway is termed No-Go because it creates a negative bias. These biases are changed by conditioning as follows: conditioning by reward produces an elevation of dopamine and thereby a strengthening of the connections of sensory cortex cells that carry cue information (see below) onto the MSN of the Go (direct) pathway that represent the rewarded action. Negative reinforcement leads to a drop in dopamine, which strengthens connections of sensory cells representing the cue onto MSN of the No-Go pathway. These dopamine-dependent modifications occur only at synapses that carry an eligibility trace. This trace may be determined by previous activity during action selection. Alternatively, as proposed here, the trace is set after action selection due to an efference copy of the selected action that is sent from premotor cortex to the striatum; e.g. the signal for L is carried to L MSN-D1 and L-MSN-D2 cells by the pathways shown in dark green (the corresponding pathways for R are not shown). Gpe, globus pallidus externus; GPi, globus pallidus interna; SNc, substantia nigra pars compacta; SNr, substantia nigra pars reticulata. (Online version in colour.)