Macrocircuits: Decision Networks

Abstract

Decision-making requires stimulus categorization and localization to guide accurate responses that can be produced through multiple effectors. The success of actions is monitored so that performance can be adjusted to achieve goals. This review will survey recent empirical and theoretical developments very selectively with an emphasis on neurophysiological data from nonhuman primates that provide the clearest information about neural mechanisms.

What is a decision?

Decision-making requires multiple processes¹. A choice is required when an organism is confronted with alternatives for which an action is required to achieve a goal. Choices are evaluated as good or bad according to whether goals are achieved and consequences are as expected. The term decision is used casually and technically in several non-interchangeable senses. In particular, we can refer to a decision as a deliberation process that results in the overt act of choosing. Decision as a process has two logically and mechanistically distinct meanings – “decide to”, which is a selection between alternative actions, and “decide that”, which is a selection between alternative categories of an stimulus or concept. The logical distinction is easy to see. Whereas you can “decide that” falsely, it is not intelligible to “decide to” falsely. “Decide to”, like choosing, is judged just as good or bad. However, important distinctions can be recognized between “choose” and “decide to”. Whereas choice refers most clearly to the final commitment to one among alternative actions, decision refers most clearly to the deliberation preceding the action. The polarity between deciding and choosing is highlighted further by appreciating that although choices can be predicted, decisions cannot be predicted. If you can say what you will decide, then you will have decided.

The semantic and logic differences between “decide that” and “decide to” are paralleled by their embodiment in different neural circuits. Neurons in sensory structures that encode the features of stimuli provide the input to circuits in association structures that accomplish the categorization and localization that constitutes the evidence for a decision. Neurons in motor structures that control the innervation of the muscles accomplish the actions guided, for better or worse, by the sequence of decision processes. Sensorimotor association structures, exemplified by the frontal eye field (FEF), consist of a diversity of types of neurons with different patterns of modulation derived from different inputs to different cortical layers. The distinction between “decide that” and “decide to” also corresponds to a very long history of experimental and theoretical psychology that describes response times as the outcome of successive stages of processing². Of course, decisions have consequences. Detecting and adapting to consequences is distinct from the actual decision process. Figure 1 summarizes this framework.

Figure 1.

Neural networks for the guidance and control of visually guided saccades. Consider visual search for a red “T” among randomly oriented red and green “L”s. The color and shape of the objects are specified in feature maps that could also represent motion, depth and other visual features. These feature maps converge on a map that represents the evidence for salience at each location. This salience map is also informed by a target template in working memory. The timecourse of the salience evidence representation at the target location (ST, solid line) and a distractor location (SD, dotted line) is plotted. According to the gated accumulator model, this evidence is integrated by a network of mutually-inhibitory units that will produce a saccade to the target (GOT, solid line) or to a distractor (GOD, dotted line). A gate (orange box) prevents integration of noise by requiring the salience evidence to be of sufficient magnitude. A saccade is produced when the activation of a GO unit reaches a threshold (gray horizontal line) at which point inhibition is imposed on omnipause (OPN) neurons (red line) that releases inhibition of burst neurons (BN) that innervate motor neurons (MN) to produce a pulse of force to rotate the eye rapidly. The eye velocity signal from the BNs are integrated by a network of tonic neurons (TN) that also innervate the MN to establish a step of force necessary to maintain eccentric fixation of the target. The activation of the GO units is also influenced by gaze-holding STOP units that release inhibition on the GO units while saccade preparation transpires. If a stop signal of some kind occurs, then the STOP units potently interrupt the GO unit activation from reaching the threshold; this interruption occurs within the theoretical interval known as stop signal reaction time (SSRT) (rightmost columns). An executive control network (yellow) comprised of neurons sensitive to errors, reward and the conflict arising from co-activation of mutually incompatible response processes signals the consequences and conditions of an action. This executive control network may influence the level of the gate that systematically changes the beginning of the accumulation process to emphasize either speed or accuracy in task performance.

Decide that – Categorization and stimulus selection

If objects in the environment are not discriminated correctly, decisions cannot be effective. Neural circuits responsible for object categorization extend from the primary sensory structures that encode basic stimulus features to association areas in parietal and frontal lobes³. One well-known line of research has investigated perceptual categorization by requiring subjects to discriminate the direction of motion of a stochastic dot display⁴. This work has revealed much about the encoding of stimulus motion by area MT and the evolution of activity of neurons in area LIP to arrive at a categorization of the motion direction⁵. More recent work has demonstrated the necessity of mastering the stimulus-response association⁶ and elaborated the differential contribution of neurons in the caudate⁷ and frontal eye field (FEF)⁸. Other work also highlights the contribution of the frontal lobe to stimulus categorization^9,10,11.

The world presents us with many stimuli, most of which must be ignored in the guidance of action. Visual search, selection of a target from among non-target objects, has also been used to study perceptual decision-making. The investigation of the neural basis of visual search has been framed by the discovery that neurons in frontal, parietal, temporal and occipital cortical areas as well as the superior colliculus and thalamus respond to target and distractor stimuli initially equivalently but then over time the activity representing the target remains elevated or increases while the activity representing distractors is attenuated¹².

Several recent studies have measured this target selection process across brain regions and measurement levels. When a target is selected, does it happen more or less simultaneously across the network or in some sequence? These studies have focused on the FEF and areas in the back of the brain. The relevance of FEF for processes occurring in the back of the brain has been demonstrated vividly by the finding in monkeys and humans that stimulation of FEF influences the allocation of visual spatial attention^13,14 perhaps through modulation of neural activity in extrastriate visual areas^15,16. The various studies provide much more information than is summarized here, and they differ in a number of crucial details that frame their interpretation¹⁷. Every study has found that when the target is more difficult to locate, neurons in the FEF signal its location before neurons in occipital, parietal and temporal areas^{18,19,20,21,22}. However, findings differ when the target is located easily (pop-out search). One study reports that the parietal cortex locates the target before the frontal cortex¹⁸, but two others find that parietal cortex signals do not precede frontal target selection^23,24. This pattern of results was obtained with intracranial recordings of spikes and LFP and also with an event-related potential component known as the N2pc²⁵ (or posterior contralateral negativity²⁶) that is believed to originate from visual and association areas in the parietal and temporal lobes^27,28.

The association between FEF and V4 was mediated primarily by visual and not visuomovement or movement neurons in FEF²⁹. The evidence that visual instead of movement neurons in FEF influence V4 is consistent with recent anatomical data showing that whereas only neurons in layer 5 of FEF project to brainstem saccade structures such as the superior colliculus, the major source of input to extrastriate visual cortical areas like V4, MT and LIP arises from layer 2–3 of FEF³⁰. Moreover, V4 and MT (and probably LIP) are innervated by different neurons in FEF that themselves have qualitatively different afferents³¹. Thus, the “top-down” signal from FEF to the back of the brain is not a single mechanism; each area is under some as-yet-to-be-determined different quality of influence. Additional indirect evidence that the influence of FEF on V4 is mediated by supragranular more than infragranular neurons is provided by a recent study showing that blocking D1 but not D2 receptors in FEF influenced V4 activity based on the lower density of D2 receptors in supragranular FEF³².

Decide to – Response selection and preparation

Locating and categorizing objects does not oblige any particular action. Other neural circuits are responsible for selecting and producing body movements to achieve goals. The neural dissociation between “decide that” circuits and “decide to” circuits has been demonstrated in numerous studies; for example, when no saccade is produced, the search target selection process transpires normally while presaccadic movement neurons are suppressed ^29,33,34.

Research over decades has shown that movements are prepared through the progressive increase of discharge rate of neurons innervating premotor structures in the brainstem and spinal cord, and movements are initiated when the discharge rate reaches a threshold that does not vary with RT^35,36,37,38. New work has suggested instead that movements are initiated when a neural population reaches some point in a high-dimensional dynamical space³⁹. Recent findings from the FEF indicate that these alternative hypotheses may not be contradictory⁴⁰.

Explaining how sensory representations lead to accurate movements is a classic problem. One approach to this problem is based on the premise that noisy evidence guiding a response is accumulated over time until a threshold is achieved at which time the response is initiated. A recent model inspired by this approach provides an explanation for how signals from neurons that represent target salience can be transformed into a saccade command^41,42. The model begins with the simple assumption that the input to the neurons producing saccade responses is simply a feed-forward cascade of the output of the visual selection neurons representing the salience at the various locations in the search array. The stochastic variability in the evidence provided by the selection process is translated into variability in choice performance through the accumulation of that evidence by a network of mutually inhibitory, leaky integrators. The evidence accumulated by the network of integrators was equated with the spike trains recorded from the visual selection neurons in the FEF. Accumulated variability in the firing rates of these neurons explains the probability and timing of correct and error responses with search arrays of different set sizes if the accumulators are mutually inhibitory. Although not designed in the model, the dynamics of the stochastic accumulators quantitatively correspond to the activity of presaccadic movement neurons that initiate eye movements if gating inhibition prevents accumulation before sufficient evidence about stimulus salience has emerged. Adjustments in the level of gating inhibition can control tradeoffs in speed and accuracy that optimize visual search performance.

Although this is the only model of visual search that accounts for response time distributions⁴³, it assumes that saccade production is guided entirely by the visual salience representation so errant saccades originate in a failure to represent evidence correctly. While this has been observed in some testing conditions^44,45, in other conditions the salient target is located correctly, but the responses are incorrect^46,47. If the evidence is correct, why was an error made? Obviously, the response production stage, while guided by, can operate independently of the perceptual stage. Indeed, response selection errors can be corrected before visual processing can register that the gaze shift was an error⁴⁸.

Monitoring consequences

The correction of errors before sensory processing can be completed demonstrates the existence of a system that monitors performance. Research over the last 20 years has characterized the role of a circuit involving medial frontal cortex in executive control for limbs⁴⁹ and eyes⁵⁰. A major thread of this research began with the discovery of the error-related negativity (ERN), an event-related potential that occurs when participants produce errors⁵¹. Macaque monkeys possess the same error monitoring system as evidenced by neural spikes^52,53 and local field potentials^54,55 modulated after errors. In fact, monkeys also exhibit the ERN⁵⁶. A diversity of other neurons signal the anticipation and delivery of feedback and reinforcement and also perhaps conflict between competing response processes⁵⁷; some of these resemble signals produced by brainstem dopamine neurons⁵⁸. The presence of these signals is consistent with models of executive function based on reward prediction error^59,60.

Adjusting performance

Stochastic accumulator models account for adaptation of RT to minimize errors and maximize rewards most commonly through changes in the amount of accumulation necessary to trigger a response^61,62,63. Recent fMRI studies have reported evidence consistent with this^{64,65,66,67,68}. However, this conclusion may be premature. First, the areas with clearest modulation were in medial frontal cortex. Current neurophysiological evidence shows that weak electrical stimulation of SEF can elevate RT⁶⁹, but individual neurons in SEF do not control directly saccade initiation⁷⁰, nor do single neurons in SMA or pre-SMA control directly limb movement times⁷¹. Therefore, medial frontal areas could contribute to strategic RT adjustment necessary for SAT, but neurophysiological evidence is inconsistent with the mapping of a response threshold on activation in these areas. Also, mapping particular parameters of very simple computational models to highly-derived measures of cerebral oxygen utilization seems uncertain.

Moreover, two recent neurophysiological studies provide compelling evidence that the accumulator models do not account for all of the adjustments that mediate speed-accuracy adaptation. One study showed that the adaptive slowing of RT in the stop signal task is accomplished not by a change of threshold, baseline, or accumulation rate, but instead through a change in the time when presaccadic movement activity first begins to accumulate⁷². Another study trained macaque monkeys to trade speed for accuracy on cue during visual search⁷³. This speed-accuracy tradeoff was accomplished through several distinct neural adjustments. When accuracy was cued, baseline discharge rate was reduced before visual search arrays appeared, visual response magnitude was attenuated, neural target selection time was delayed, and movement-related activity accumulated more slowly to a lower level before saccades. This surprising pattern of modulation demonstrates that the popular stochastic accumulator models do not provide an accurate or complete description of how the brain adjusts performance.

Summary

Decision-making is accomplished by a diversity of neural circuits that influence one another is ways that remain poorly understood. Computational models of decision-making are simplifications that embody particular assumptions based on intuitions about simplicity and optimality. These models have proven very effective at describing performance on decision tasks. The evident parallels between the form of activity of some neurons and the form of the processes in these models has invited and encouraged rather direct mapping of model process onto particular neurons. However, new data indicate that correct formulation of such linking propositions will require an unexpected degree of subtlety and nuance. For example, models that require discrete stage completion times are inconsistent with the diversity of times when stimulus categorization and localization is accomplished in different structures. Also, models that explain all of the variation of error and correct responses in a single evidence-accumulating categorization stage are inconsistent with the production of errors by the response selection circuit. How to translate between different levels of analysis and description remains a major challenge. The heterogeneity and anatomical specificity of neural circuits, timings and processes (that cannot be resolved by measures like ERP and fMRI) must be appreciated to evaluate how models of decision-making actually map onto neural mechanisms.

Highlights.

• Categorization and stimulus selection are accomplished by sensory and association circuits

• Response preparation is accomplished by association and motor circuits

• Monitoring and adjustment involve medial frontal areas

• Speed-accuracy adjustments involve perceptual and response stages