Comparing the Bird in the Hand with the Ones in the Bush

Abstract

In this issue of Neuron, Boorman and colleagues shed new light on the roles of lateral frontopolar and ventromedial prefrontal cortices in task switching and decision making.

A Nightingale, sitting aloft upon an oak, was seen by a Hawk, who made a swoop down, and seized him. The Nightingale earnestly besought the Hawk to let him go, saying that he was not big enough to satisfy the hunger of a Hawk, who ought to pursue the larger birds. The Hawk said: “I should indeed have lost my senses if I should let go food ready to my hand, for the sake of pursuing birds which are not yet even within sight.”

—Aesop’s Fables: A New Revised Version from Original Sources (translator not identified), 1884

So goes the fable of Aesop, versions of which have been transmitted around the world for over 2500 years. Indeed, a bird in the hand is worth two in the bush—unless, perhaps, the probability of catching the birds in the bush is very high. In a dynamic environment, it is adaptive to monitor the possible outcomes associated with alternative courses of action and to update our behavior accordingly.

In the current issue of Neuron, Boorman, Behrens, Woolrich, and Rushworth provide compelling evidence for a neural mechanism by which this monitoring of alternative outcomes relative to current outcomes, and the subsequent updating of behavior, can occur (Boorman et al., 2009). In particular, they show that our brains can keep track of the mounting evidence in favor of an alternative course of action, and that—when strong enough—this signal leads to a switch in behavior. The evidence favoring a switch to an alternative choice is tracked by lateral fronopolar cortex (FPC), and this information appears to be transmitted to the inferior parietal sulcus area (IPS) and ventral premotor cortex (PMv) in advance of a behavioral switch. By contrast, the immediate relative value of the current choice is encoded by ventro-medial PFC (vmPFC).

The approach of Boorman and colleagues was to combine a very simple decision-making task with sophisticated mathematical modeling and functional magnetic resonance imaging (fMRI) in humans. Participants selected one of two responses (a left or right button press) on every trial. The potential reward for each option was shown, and these rewards varied randomly from trial to trial. The probability of reward for each option was unknown but could be estimated from experience. Optimal estimates of these probabilities were generated by a Bayesian model that tracked the long-term probability of each option, as well as the volatility of these probability estimates, based on observed outcomes. By manipulating reward values on the basis of prior selections, the value associated with an option (i.e., probability 3 reward) was made to be independent of reward probability. As such, the authors were able to separate long-term probability estimates associated with competing options from the immediate value computations that drive decision making.

This important study has several main findings. First, activation in bilateral FPC tracks the probability that the unchosen option will be rewarded, relative to the probability that the chosen option will be rewarded—i.e.,the relative unchosen probability. Critically, when FPC activation between trials was high, it was likely that a participant would switch from one option to the other on the subsequent trial. In addition to these within-subject correlations between activation and choice behavior, the authors found significant, albeit more modest, between-subject correlations, such that subjects with greater FPC engagement for relative un-chosen probability were more likely to make advantageous switches. Based on these findings, the authors conclude that FPC keeps track of the cumulative evidence in favor of a switch in behavior.

The experiment was set up such that tracking the probability of reward for each response was useful for decision making, whereas tracking reward magnitudes from trial to trial was not. It is therefore an open question whether or not FPC would track average reward if the task were modified to make this variable relevant. If FPC were shown to track average reward as well as probability, then this would strongly support the general conclusion that this region tracks the relative advantage of switching, rather than the more narrow interpretation that it only tracks reward probabilities.

Like FPC, the IPS was engaged by relative unchosen probability, but only on switch trials, in which the previously un-chosen option becomes the chosen one. This finding suggests that the signal providing evidence for a switch arises in FPC and is transmitted to the IPS. Consistent with this idea, right IPS and right premotor cortex demonstrate increasing functional connectivity with FPC with increasing relative unchosen probability on switch trials. These findings provide insight into a brain mechanism whereby behavior can be updated flexibly in response to a changing environment.

The authors provide evidence for two distinct neural signals: one produced by FPC that provides evidence in favor of the alternative outcome and one produced by vmPFC that encodes the value (immediate reward 3 long-term probability) of the chosen option relative to that of the unchosen option (i.e., “relative chosen value”). That vmPFC tracks value has been shown before (Hare et al., 2008), but the finding that this signal is relative (chosen value minus unchosen value) is novel. The authors argue convincingly that vmPFC encodes the value-based evidence in favor of the selected option over the course of multiple decisions.

These intriguing results provide fresh insights into the neural mechanisms of decision making and especially into the function of FPC, a region that has received intense scrutiny over the last few years. Boorman and colleagues’ findings are consistent with the idea that FPC accumulates evidence in favor of switching and then with sufficient evidence provides a signal to indicate that switching should occur.

To situate these findings relative to prior studies of FPC, the authors suggest that this region plays a more general role in representing pending or alternative courses of action. In other words, FPC may encode the alternative option and not simply the evidence for it. In this case, the representation of a favored option would involve greater neural activity than the representation of a less favored option. The possibility that FPC represents alternative options is broadly consistent with theories on multitasking and prospective memory, which posit that FPC stores information that is needed for later performance (Koechlin and Hyafil, 2007; Burgess et al., 2003).

An alternative account of lateral frontopolar function posits that this region is involved in the joint consideration or integration of distinct mental representations (Christoff et al., 2001; Ramnani and Owen, 2004). Studies involving Raven’s Progressive Matrices, relational matching tasks, propositional analogies, and transitive inference problems have shown that FPC is more active on trials that require integration or comparison of two relations between sets of mental representations relative to the separate consideration of two such relations (Christoff et al., 2001; Bunge et al., 2005; Smith et al., 2007; Wendelken and Bunge, 2009). A role in the comparison of distinct representations can also account for FPC activation in studies in which a mental representation must be evaluated according to specific criteria, as in episodic memory judgments. It should be noted that relational integration, as well as episodic retrieval, typically involves only lateral frontopolar cortex, whereas multitasking frequently activates more medial parts of the frontal pole (Gilbert et al., 2006). In the current experiment, FPC can be interpreted as engaging in the active comparison of competing response options. A comparison process would be expected to intensify as the evidence for switching increases, and a comparator circuit should interact with other brain regions involved in switching, just as an evidence accumulator would. Whether or not this interpretation is accurate, the present findings should lead to the refinement of various accounts of FPC function.

Given the evidence of strong functional connectivity between FPC and IPS in the present study, as well as in a study of exploration versus exploitation in decision making (Daw et al., 2006), and in our own research, it is reasonable to ask whether there might be a direct anatomical connection between these regions in humans. Boorman and colleagues note the evidence for a lack of connection between medial frontopolar cortex and parietal cortex in macaque monkeys and speculate that the connection between FPC and IPS may be mediated by PMv or by dorsolateral PFC. However, the possibility of a direct anatomical connection should not be discounted, given the identification of a new subregion of Brodmann area 10 in humans relative to nonhuman primates (Ongür et al., 2003) and the marked differences in activation profiles of FPC as compared with medial frontopolar cortex (Gilbert et al., 2006). Perhaps the strengthening of connections between FPC and IPS in humans contributes to the uniquely human capacity for flexibly updating behavior as a function of changing environmental context (Stoet and Snyder, 2003).