Neural underpinnings of the evidence accumulator

Abstract

Gradual accumulation of evidence favoring one or another choice is considered a core component of many different types of decisions, and has been the subject of many neurophysiological studies in non-human primates. But its neural circuit mechanisms remain mysterious. Investigating it in rodents has recently become possible, facilitating perturbation experiments to delineate the relevant causal circuit, as well as the application of other tools more readily available in rodents. In addition, advances in stimulus design and analysis have aided studying the relevant neural encoding. In complement to ongoing nonhuman primate studies, these newly available model systems and tools place the field at an exciting time that suggests that the dynamical circuit mechanisms underlying accumulation of evidence could soon be revealed.

Introduction

When we face a difficult decision, and are therefore uncertain as to what the best choice is, we are slow to make up our minds; but when faced with an easy decision, we are fast. This experience from daily life is one of the most common behavioral observations in decision-making, and applies in a remarkably wide array of different types of decisions, ranging from perceptual decisions [1], to numerical comparison decisions [2], to social decisions [3], to visual search decisions [4,5], to gambling decisions [6], to memory retrieval decisions [7], to lexical retrieval decisions [8], to social decisions [3], to value-based decisions [9–15]. A conceptually simple model, introduced many decades ago in the behavioral literature [7,16–19] has been able to account very well for the observation across all the above decision-making domains. As a result, this model, known as the “evidence accumulation” or “evidence integration” model, has become widely adopted as a succinct yet powerful behavioral-level description of core decision-making processes.

The central idea of the model is that as a subject is forming a decision, evidence for or against different possible choices is gradually accumulating in the subject’s mind; the final value of this accumulated evidence then drives the decision itself-- for example, committing to a particular choice by asking whether the accumulated evidence lies to one side or another of a reference value which we will label as the “decision boundary” (see Fig. 1a). In this model, when the evidence is strong, the accumulator’s value quickly diverges away from the decision boundary, and it rapidly become easy to say on which side of the decision boundary it is. Whereas when the evidence is weak, the value of the evidence accumulator meanders away from the decision boundary only very gradually, leading to slower, more difficult decisions.

Figure 1.

Evidence accumulator models and associated circuits. a, Schematic of evidence accumulation process, here illustrated for a case when the subject must decide between orienting Left or Right. As the decision process unfolds, noisy evidence favoring one choice (RIGHT) adds to the accumulator while evidence favoring the other choice (LEFT) subtracts from the accumulator. The sign of the accumulated evidence when the subject is asked to report their decision dictates the resulting decision choice. Trials with strong evidence that more consistently favors one choice over the other result in steeper slopes on average, and the accumulator will soon be far away from the decision boundary, so easy decisions can be made quickly. Weaker, less consistent evidence will result in meandering trajectories with shallower slopes on average, and even after lengthy accumulation periods, the accumulator may not be far from the decision boundary, leading to slow, more error-prone decisions. In tasks in which the subject determines the duration of the decision process, known as “reaction time tasks,” the subject commits to a decision when the evidence reaches a bound (+C or −C in the figure); the reaction time is determined by when the bound is reached, and the decision choice is given by which bound was reached. b, Average neural responses from monkey PPC (area LIP) during the period of decision formation in the random dot motion discrimination task [27]. After a delay, responses exhibit ramping response profiles with slopes that depend on stimulus strength. Stronger motion leads to sharper slopes and weaker motion to shallower slopes. This corresponds to the average trends predicted by the evidence accumulator model. c, Diagram of interconnected brain regions that have been demonstrated to exhibit responses profiles correlated with accumulating evidence. These areas thus serve as candidates to be involved in the evidence accumulation process.

Here we will first briefly describe studies with non-human primates into the neural basis of the evidence accumulator. Our main focus will then be on more recent work using rodents, proposed as a complementary model system with which to unravel the mechanistic circuit dynamics underlying the phenomenon. We limit ourselves to two-alternative decisions (for multi-alternative decisions, see [20]), and will not address work in humans (see [21]).

Seminal studies in non-human primates

Starting in 1996, Michael Shadlen, William Newsome and colleagues (in addition to related parallel work from Jeffrey Schall’s group [22]), began a series of highly influential electrophysiological experiments that suggested a connection between neural responses and evidence accumulation. Using monkeys trained to perform a visual perceptual decision-making task in which the experimenters could titrate each trial’s difficulty (“random dot motion discrimination” (RDM) task; [23,24], Shadlen and Newsome found that during the process of decision formation, neurons in the lateral intraparietal (LIP) subregion of posterior parietal cortex (PPC) had firing rates that appeared to ramp up in time. Critically, when averaged across easy trials, firing rates ramped slowly; but when averaged across difficult trials, firing rates ramped much more sharply--precisely as expected of the evidence accumulator ([25–27]; Fig. 1b). This was the first time anyone had observed a signal inside the brain that matched what had been predicted for many years by the widespread accumulator model. Their seminal finding led to the proposal that there may be a 1-to-1 relationship between PPC firing rates and the value of the evidence accumulator.

Work in several laboratories (e.g., [28]) has uncovered similar firing rate patterns in multiple brain regions, most prominently in the frontal eye fields (FEF) [4,29–31], but also in other regions (dorsolateral prefrontal cortex, [29]; superior colliculus [32–34]; and striatum [35]; Fig. 1c). To date, causal perturbation studies of these areas with the primate RDM task have been limited, with only three existing published studies, all using only unilateral electrical microstimulation. Gold and Shadlen used microstimulation in the FEF to conclude that “developing oculomotor commands may reflect the formation of the monkey's direction judgement,” but made no conclusions about the causal role of the FEF itself [36,37]. In the striatum, Ding and Gold found mixed effects, with contralateral responses speeded up even while, surprisingly, ipsilateral responses were favored [38]. Finally, in the PPC, Hanks et al. found that whole-trial microstimulation could be interpreted as adding a constant offset to the accumulator variable [39], although they acknowledged that their data “cannot rule out the possibility that some of the observed effects are due to antidromic activation of other areas.” This concern applies to all microstimulation studies, and is consistent with the later claim that microstimulation acts mainly through activation of axons [40] (but see [41]). The antidromic activation issue is not a concern with more recent inactivation studies to be described below. Despite these many studies, unraveling the neural circuit mechanisms underlying the accumulation process has remained a challenge, and major fundamental questions are still unanswered or even unaddressed. Which brain regions are necessary or not necessary for the evidence accumulation process? Where in the brain is evidence accumulation actually computed--perhaps in brain region(s) not yet recorded from? What is the macro- and micro- circuit architecture that supports the gradual accumulation computation?

New tools and animal models for studying gradual evidence accumulation

To assist in addressing these questions, Brunton et al. developed a rat model of a perceptual decision task dependent on gradual accumulation of evidence (“Poisson clicks” task, [42]; see also [43,44]). Rats were presented with two simultaneous trains of randomly timed auditory clicks, one to their left, the other to their right, and were trained to orient at the end of the stimulus towards the side that had the greater total number of pulses. As with the primate RDM task, trial difficulty could be titrated by the experimenters, in this case by controlling the left : right ratio of the Poisson rates that generated the random pulses. The highly variable yet very precisely known timing of the pulses ensured that stimulus space was thoroughly explored. It provided rich information that gave statistical power to a trial-by-trial decision process model that took into account the timing of each individual click, and that allowed estimating multiple parameters of the decision process. Together with model-free analyses that supported the main conclusions, the model provided strong behavioral evidence that the rats were indeed using gradual accumulation of evidence to perform the task. The door was open for using a cheap, small, tractable mammalian animal model for studies of evidence accumulation.

Using rats trained to perform the Poisson clicks task, Hanks, Kopec, et al. recorded from two cortical regions in the rat brain, the rat PPC and the frontal orienting fields (FOF; [45]). On the basis of their connectivity with other brain regions and their physiological properties, these had been suggested as analogous or perhaps even homologous to the two key primate regions PPC and FEF, respectively [46,47]. As in the primate, Hanks, Kopec, et al. found that neurons in the rat regions had trial-averaged ramping firing rates during the process of decision formation. Further, the slope of the ramp was steeper for stronger evidence-- again, just as had been observed in the primate brain regions. Also as in the primate, traces were qualitatively similar across the two recorded regions (Fig. 2a,b). The similarities across species suggested that rats and primates might be using similar circuits and algorithms to solve the task. Whether this is indeed the case remains an open question, of course. But it became clear that a fruitful comparison and intellectual exchange across model species was possible.

Figure 2.

Characteristics of rat PPC and FOF during accumulation of evidence based decision making, from [45]. a, Average neural responses in rat PPC during the Poisson Clicks task. Trials are grouped by average stimulus strength. Similar to monkey PPC, responses exhibit ramping profiles that depend on stimulus strength. b, Same as a for FOF. c, Click-triggered average responses for rat PPC during the Poisson clicks task. Individual clicks have a measurable and sustained influence on responses in PPC. d, Same as c for FOF. Individual clicks also produced a sustained response, with a magnitude that slowly but significantly decayed over hundreds of milliseconds. e, Time-average population comparison of tuning curves for accumulating evidence in PPC and FOF. PPC shows a smoothly graded relationship, while FOF shows a sharper dependence on the sign of the accumulator value. f, Bias caused by 500-ms unilateral inactivation of FOF with halorhodopsin during one of four epochs of the task: before the stimulus (red), during the first half of the stimulus (yellow), during the second half of the stimulus (green), or during the movement period (blue). Only peri-choice perturbation of FOF has a significant effect on decision making. A further experiment using half the inactivation time period (250 ms) reached the same conclusion [45].

The pulsatile nature of the Poisson click stimulus, combined with a model-based estimate of each trial’s evolution of the accumulator, allowed Hanks, Kopec, et al. to perform two analyses that in previous primate work had required additional experiments. First, by computing the click-triggered average firing rate, it was possible to estimate the impact of a single evidence pulse on a neuron’s firing rate. A perfect accumulator would respond with a permanent change in its value. Both PPC and FOF were found to respond to an evidence pulse with sustained changes in firing rate, albeit with a slow decay in the case of the FOF (see Fig. 2c,d and see [48] for the corresponding primate PPC experiment). Second, the Brunton et al. model produced a moment-by-moment estimate of the gradually evolving value of the evidence accumulator. This meant that at each point in time, in each trial, Hanks et al. had both a measure of a neuron’s firing rate and an estimate of the value of the evidence accumulator. Collating these paired measures across trials allowed building “tuning curves,” plots of average firing rates as a function of the variable of interest, which in this case is the value of the accumulator.

We note that this model-based approach combines firing rate measurements with knowledge of the full, detailed within-trial dynamics of the sensory stimulus as well as the animal’s behavior. It is thus very different to a variety of interesting methods that have focused entirely on statistical analyses of neuronal firing [28,49,50]. Future work combining both types of approaches will be valuable.

The new tuning curve analysis tool revealed that, on average across the population, tuning curves in both the PPC and the FOF were relatively stable during the decision formation. Thus, although the value of the accumulator changes during the course of a trial, the mapping from accumulator value to firing rate does not. But while tuning curves in PPC were found to be smooth functions of the value of the accumulator --indicating that the graded value of the accumulator could be read out from PPC firing, as also found in the primate PPC [51]-- the tuning curves in the FOF were much more step-like, with one cluster of firing rates for accumulator values to one side of the decision boundary, and a different cluster of firing rates for values on the other side of the decision boundary (see Fig. 2e). Thus what is best read out from FOF firing is not the graded value of the accumulator, but rather which categorical decision report the subject should make if the trial ended at that point in time. This suggests that the FOF may be more strongly involved in categorizing the value of the accumulator to drive the final choice than in computing the graded accumulation itself. The tuning curve analysis thus allowed distinguishing two regions with qualitatively similar responses, and began the task of differentiating which, out of the various steps in decision formation, each region might be more closely associated with. More specifically, the analysis suggested a simple serial organization in our fully trained rats, in which the graded value of the accumulator is computed in the PPC, and then read out from there to be turned into a more categorical representation in the FOF; which in turn eventually drives the categorical motor acts with which the animal reports its decision. In this serial model, activity in both regions is critical for task performance, but activity in the FOF is required specifically near the time when the animal is triggered to report its decision (the “GO cue”), since that is when the overall decision process needs to read out on which side of the decision boundary the accumulator value lies. As we describe below, this serial model turned out to be wrong.

Perturbation studies

One key advantage of rodents is that computerizing and parallelizing behavioral training across subjects can produce a sufficient number of trained animals to make multiple perturbation experiments viable. By their very nature, perturbation experiments alter the brain, and therefore tend to be used very sparingly when individual animals are expensive in terms of cost, time, or effort involved in their training. For example, producing 50 trained subjects to troubleshoot and deploy a new perturbation technique can be prohibitively expensive with non-human primates. In contrast, it is straightforwardly practical with computerized, parallelized rodent training.

The tens of milliseconds time resolution afforded by optogenetic inactivation, in this case using the virally-delivered, light-activated chloride pump eNpHR3.0 [52] injected unilaterally into the FOF, allowed probing and confirming the hypothesis about the specific times when activity in the FOF would be required (see Fig 2f; [45]). Model-based analysis of performance impairments caused by unilateral, as well as bilateral, hours-long pharmacological silencing of the FOF [53] provided further converging evidence for the hypothesized role for the FOF: namely, that the FOF’s main role in the task lies in categorizing the value of the accumulator, an operation that occurs after the graded evidence accumulation itself. Ongoing experiments are further probing the hypothesis through temporally-specific bilateral optogenetic inactivation of the FOF.

In contrast to expectations, however, inactivation of the PPC produced entirely negligible effects on performance of the Poisson Clicks task [53]. This was the case even while the same inactivations had a large effect on interspersed control “free choice” trials [53], and even though PPC inactivations have a substantial effect on a different auditory task involving parametric working memory [54]. These data suggest that despite its encoding of the graded value of the accumulator, the PPC may play little to no role in decisions driven by accumulation of evidence. Supporting this idea, preliminary data reproduces the PPC inactivation finding in visual evidence accumulation tasks in primates [55], inactivation of the primate pulvinar, which is the PPC-projecting region of the thalamus, has no impact on decision choices in a related visual task [56], inactivation of the primate PPC has no effect on visual primate memoryguided tasks [57–60], inactivation of mouse PPC has no effect on a somatosensory memory-guided task [61], and temporally-specific inactivation of mouse PPC in a visual memory-guided task has an effect only during the sensory stimulus period, not the short-term memory maintenance period [62]. While the PPC could, perhaps, play a role in gradual accumulation as part of a much larger redundant circuit in the Poisson Clicks task, it is notable that out of 6 different rat regions probed so far (medial prefrontal cortex and superior colliculus, (Hanks, Yartsev, et al. unpublished data); anterior dorsal striatum and auditory striatum (Yartsev, Hanks, et al., unpublished data); and FOF and PPC), the only region for which silencing has no impact on task behavior is the PPC. Consequently, we consider the PPC to have the lowest likelihood of being an important center of the causally relevant circuit for gradual evidence accumulation.

In contrast to primate visual tasks and the rodent auditory Poisson Clicks task, but consistent with Goard et al.’s preliminary optogenetic inactivation data [62], hours-long pharmacological inactivation of the PPC does impair rodent visual short-term memory or evidence accumulation tasks [63,64]. Immediately posterior to the rodent PPC are a set of individually small visual areas, collectively referred to as secondary visual cortex (V2; [65–67]). With the precise definition of rodent PPC and the location of its border with these V2 areas still highly uncertain and a matter of active research [68–70], the observed effects from inactivations targeting rodent PPC could perhaps be due to inactivation spillover into one of the many small V2 areas. The appeal of this possibility is that it would make results across mouse, rat, and monkey consistent, for the difference in results between rodent visual tasks versus primate visual tasks or rodent non-visual tasks would be explained by the fact that spillover into the relevant sensory cortices is not a concern in the latter two cases. Even if this view were correct, the role of any such V2 sub-area in accumulation of evidence tasks remains to be determined and could not simply be basic sensory processing of visual stimuli [63]. Perhaps like the FOF, which we have posited as being required for reading the output of the accumulator, this region might be required for providing visual sensory input to the accumulator, even while not being involved in accumulation itself.

Further clarity on the role of the PPC will greatly benefit from better definitions and knowledge about the topography of brain regions near the location currently referred to as “rodent PPC.” We advocate reserving the name “posterior parietal cortex (PPC)” in rodents to regions displaying physiological and connectivity profiles similar to those of the primate PPC, for example, no impact from inactivation on visual accumulation of evidence tasks [55], little anatomical input from primary visual cortex [71,72], and strong connectivity with frontal regions, as is the case with the anterior, as opposed to the posterior, zones currently referred to as “rat PPC” [70].

So where is the accumulator circuit? Preliminary hints and research outlook

To study the macro- and micro-circuitry causally responsible for gradual accumulation, we must first know which brain regions are part of that circuitry. A region that is part of the relevant causal circuit would be expected to simultaneously satisfy three initial criteria: (1) inactivation of the region should have an effect on task performance. More specifically, because there are different sequential steps involved in the decision formation and gradual accumulation is only one of them, (2) perturbation during temporally-specific periods that coincide with the evidence accumulation period should impact performance. And (3) the graded value of the accumulator should be encoded in that region’s neural activity (as assessed, for example, through the tuning curve method of Hanks, Kopec, et al.). As described above, neither the FOF nor the PPC satisfy all three criteria. Where in the brain are the region(s) that do?

The data analytic and experimental tools developed to study the PPC and the FOF are now being applied to several rat brain regions, an endeavour facilitated by high-throughput rodent training. Current unpublished data (Yartsev, Hanks, et al.) suggests that while some subcortical regions may be like the FOF in playing an important role in the task, but not in gradual accumulation per se, at least one other subcortical region appears to satisfy all three criteria, and may thus become the first identified node in the causal circuit for evidence accumulation. Assuming that this preliminary conjecture is correct, by following projections to, and projections from, the identified region, we may finally be able to delineate and establish the relevant causal circuit. Initial suggestions indicate that this may include brain regions never before examined in the context of accumulation of evidence.

We are thus in a particularly exciting time for research into the neural underpinnings of the evidence accumulator: once the relevant causal circuit is established, we will at long last be able to probe, and empirically distinguish, the many theoretical accounts that have been proposed [73–86]. In addition, methods to apply cellular-resolution imaging of neural activity to this problem, with which to powerfully examine multi-neuronal population codes [87], have recently been developed [88,89], and the many genetic tools available in mice may soon be brought to bear as well [90–92].

Finally, we emphasize our strong view that the developments in rodent model species described in this review, while very exciting, are complementary to efforts in non-human primates, in which sophisticated behavioral manipulations can be most powerfully employed to reveal important aspects of neural encoding and function, very fine control over both sensory and motor responses can be achieved in the visuomotor domain (e.g., [37,93]), and which are an important link towards relating neurophysiological findings to neural mechanisms in humans.

Highlights.

• Gradual accumulation of evidence, core to decision-making, can be studied in rodents

• Pulse-based sensory evidence stimuli facilitate behavioral and neural analysis

• High-time-resolution inactivation helps identify role of different brain regions

• Differential role of the five brain regions studied so far is being distinguished

• Causal circuit for accumulation involves only some of the regions studied so far