Risky business: the neuroeconomics of decision making under uncertainty

Abstract

Many decisions involve uncertainty, or imperfect knowledge about how choices lead to outcomes. Colloquial notions of uncertainty, particularly when describing a decision as ‘risky’, often carry connotations of potential danger as well. Gambling on a long shot, whether a horse at the racetrack or a foreign oil company in a hedge fund, can have negative consequences, but the impact of uncertainty on decision making extends beyond gambling. Indeed, uncertainty in some form pervades nearly all our choices in daily life. Stepping into traffic to hail a cab, braving an ice storm to be the first at work, or dating the boss’s son or daughter also offer potentially great windfalls, at the expense of surety. We continually face trade-offs between options that promise safety and others that offer an uncertain potential for jackpot or bust. When mechanisms for dealing with uncertain outcomes fail, as in mental disorders such as problem gambling or addiction, the results can be disastrous. Thus, understanding decision making—indeed, understanding behavior itself—requires knowing how the brain responds to and uses information about uncertainty.

The economics of uncertainty

‘Uncertainty’ has been defined in many ways for many audiences. Here we consider it the psychological state in which a decision maker lacks knowledge about what outcome will follow from what choice. The aspect of uncertainty most commonly considered by both economists and neuroscientists is risk, which refers to situations with a known distribution of possible outcomes. Early considerations of risk were tied to a problem of great interest to seventeenth-century intellectuals; namely, how to bet wisely in games of chance. Blaise Pascal recognized that by calculating the likelihood of the different outcomes in a gamble, an informed bettor could choose the option that provided the greatest combination of value (v) and probability (p). This quantity (v × p) is now known as ‘expected value’.

Yet expected value is often a poor predictor of choice. Suppose that you are a contestant on the popular television game show Deal or No Deal. There are two possible prizes remaining: a very large prize of $500,000 and a very small prize of $1. One of those rewards—you do not know which!—is in a briefcase next to you. The host of the game show offers you $100,000 for that briefcase, giving you the enviable yet difficult choice between a sure $100,000 or a 50% chance of $500,000. Selecting the briefcase would be risky, as either a desirable or undesirable outcome might occur with equal likelihood. Which do you choose? Most individuals faced with real-world analogs of this scenario choose the safe option, even though it has a lower expected value. This phenomenon, in which choosers sacrifice expected value for surety, is known as risk aversion. However, the influence of risk and reward on decision making may depend on many factors: a sure $100,000 may mean more to a pauper than to a hedge fund manager. Based on observations such as these, Daniel Bernoulli¹ suggested that choice depends on the subjective value, or utility, of goods (u), which leads to models of choice based on ‘expected utility’ (that is, u × p). When outcomes will occur with 100% probability (“Should I select the steak dinner or the salad plate?”), people’s choices may be considered to reflect their relative preferences for the different outcomes².

Although expected utility models provide a simple and powerful theoretical framework for choice under uncertainty, they often fail to describe real-world decision making. Across a wide range of situations—from investment choices to the allocation of effort—uncertainty leads to systematic violations of expected utility models³. In the decisions made by real Deal or No Deal contestants in several countries, contestants’ attitudes toward uncertainty were influenced by the history of their previous decisions⁴, not just the prizes available for the current decision. Moreover, many real-world decisions have a more complex form of uncertainty because the distribution of outcomes is itself unknown. For example, no one can know all of the consequences that will follow from enrolling at one university or another. When the outcomes of a decision cannot be specified, even probabilistically, the decision is said to be made under ambiguity, following concepts introduced by Knight⁵ and the terminology of Ellsberg⁶. In most circumstances, people are even more averse to ambiguity than to risk alone.

Economists and psychologists have studied how these different aspects of uncertainty influence decision making. This research indicates that people are generally uncertainty averse when making decisions about monetary gains, but uncertainty-seeking when faced with potential losses. However, when either probabilities or values get very small, these tendencies reverse. Thus, the same individual may buy lottery tickets in the hope of a large unlikely gain, but purchase insurance to protect against an unlikely loss. To account for these uncertainty-induced deviations from expected utility models, Tversky and Kahneman proposed prospect theory^7,8, which posits separate functions for how people judge probabilities and how they convert objective value to subjective utility. Other theorists have developed models that account for the effects of ambiguity on choice, by treating ambiguity as a distribution of probabilities (and thus converting it to risk)⁹ or by modeling the psychological biases that ambiguity induces (such as attention to extreme outcomes)^10,11.

An important area of ongoing research is the study of individual differences in decision making under uncertainty. To address this issue, some researchers have evaluated whether risk attitudes constitute a personality trait^12-14. In part, these efforts reflect the intuition of both the public and the scientific community that some individuals are inherently risk-seeking, while others are consistently risk averse. Despite its plausibility, the identification of a risk-seeking phenotype has foundered on a number of difficult problems. First, risk taking seems to be highly domain specific, such that one might find very different attitudes toward risk taking in financial versus health versus social situations. For example, among 126 respondents, no person was consistently risk averse in all five content domains, and only four individuals were consistently risk-seeking¹⁴. Moreover, the wording of questions leads to systematic differences in apparent willingness to take risks. People may differ more across domains in how they perceive risk rather than in their willingness to trade increased risk for increased benefits. Despite these inconsistencies, some data suggest stable gender and cultural differences in attitudes toward uncertainty^15,16. Women, for instance, are more averse to uncertainty in all domains except social decision making¹⁴. That the same individual may express different attitudes toward uncertainty under different circumstances points to a potentially important role for neuroscience in that an understanding of mechanism may provide a powerful framework for interpreting these diverse behavioral findings.

Risk sensitivity in nonhuman animals

The canonical perspective on decision making in nonhuman animals is that, like people, they are generally risk averse. Indeed, risk aversion is reported for animals as diverse as fish, birds and bumblebees^17,18. However, recent studies provide a more nuanced and context-dependent picture of decision making under risk. For example, risk preferences of dark-eyed juncos—a species of small songbird—depend on physiological state¹⁹. Birds were given a choice between two trays of millet seeds: one with a fixed number of seeds and a second in which the number of seeds varied probabilistically around the same mean. When the birds were warm, they preferred the fixed option, but when they were cold, they preferred the variable option. The switch from risk aversion to risk seeking as temperature dropped makes intuitive adaptive sense given that these birds do not maintain energy stores in fat because of weight limitations for flight. At the higher temperature, the rate of gain from the fixed option was sufficient to maintain the bird on a positive energy budget, but at the lower temperature energy expenditures were elevated, and the fixed option was no longer adequate to meet the bird’s energy needs. Thus, gambling on the risky option might provide the only chance of hitting the jackpot and acquiring enough resources to survive a long, cold night. In humans, wealth effects on risk taking might reflect the operation of a similar adaptive mechanism, promoting behaviors carrying an infinitesimal, but nonzero, probability of a jackpot. However, state-dependent variables such as energy budget or wealth seem likely to influence decision making in different ways in different species, or even among individuals within the same species, depending on other contextual factors^20,21.

When reward sizes are held constant, but the delay until reward is unpredictable, animals generally prefer the risky option²². This behavior may reflect the well known effects of delay on the subjective valuation of rewards, a phenomenon known as temporal discounting²³. A wealth of data from studies of interval timing behavior indicates that animals represent delays linearly, with variance proportional to the mean²⁴. This internal scaling of temporal intervals effectively skews the distribution of the subjective value of delayed rewards, thereby promoting risk-seeking behavior¹⁷. Together, these observations suggest that uncertainty about when a reward might materialize and uncertainty about how much reward might be realized are naturally related. One common explanation for the generality of temporal discounting is that delayed rewards might be viewed as risky, thus leading to preference for the sooner option in intertemporal choice tasks²⁵.

The converse might also be true^26,27; for example, when an animal makes repeated decisions about a risky gamble that is resolved immediately, that gamble could also be interpreted as offering virtually certain but unpredictably delayed rewards. In a test of this idea, rhesus macaques were tested in a gambling task when there were different delays between choices²⁸. Monkeys preferred the risky option when the time between trials was between 0 and 3 s, but preference for the risky option declined systematically as the time between choices was increased beyond 45 s. One explanation is that the salience of the large reward, and the expected delay until that reward could be obtained, influenced the subjective utility of the risky option. According to this argument, monkeys prefer the risky option because they focus on the large reward and ignore bad outcomes—a possibility consistent with behavioral studies in humans and rats²⁷. Alternatively, monkeys could have a concave utility function for reward when the time between trials is short, which becomes convex when the time between trials is long. In principle, these possibilities might be distinguished with neurophysiological data.

Neuroeconomics of decision making under uncertainty

Probability and value in the brain

The extensive economic research on decision making under uncertainty leaves unanswered the question of what brain mechanisms underlie these behavioral phenomena. For example, how does the brain deal with uncertainty? Are there distinct regions that process different forms of uncertainty? What are the contributions of brain systems for reward, executive control and other processes? The complexity of human decision making poses challenges for parsing its neural mechanisms. Even seemingly simple decisions may involve a host of neural processes (Fig. 1). A powerful approach has been to vary one component of uncertainty parametrically while tracking neural changes associated with that parameter, typically with neuroimaging techniques. Such research has identified potential neural substrates for probability and utility.

Figure 1.

Brain regions implicated in decision making under uncertainty. Shown are locations of activation from selected functional magnetic resonance imaging studies of decision making under uncertainty. (a) Aversive stimuli, whether decision options that involve increased risk or punishments themselves, have frequently been shown to activate insular cortex (INS)³³,⁵²,⁵³,⁵⁸ and ventrolateral prefrontal cortex (vlPFC)⁶¹. (b) Unexpected rewards modulate activation of the striatum (STR)⁴³,⁴⁶,⁵³,⁵⁹,⁷⁶, particularly its ventral aspect, as well as the medial prefrontal cortex (mPFC)⁴³,⁵³,⁶¹,⁷⁶. (c) Executive control processes required for evaluation of uncertain choice options are supported by dorsolateral prefrontal cortex (dlPFC)⁵²,⁵⁸ and posterior parietal cortex (PPC)³³,³⁴. Each circle indicates an activation focus from a single study. All locations are shown in the left hemisphere for ease of visualization.

If a decision maker cannot accurately learn the probabilities of potential outcomes, then decisions may be based on incomplete or erroneous information. In functional magnetic resonance imaging (fMRI) experiments^29,30, subjects made a series of decisions under different degrees of uncertainty (from 60% to 100% probability that a correct decision would be rewarded). Importantly, subjects were never given explicit information about these probabilities, but learned them over time through feedback from their choices. Activation of the dorsomedial prefrontal cortex (Brodmann area 8) was significantly and negatively correlated with reward probability, an effect distinct from the activation associated with learning about probabilities. The medial prefrontal cortex has been previously implicated in other protocols in which subjects learn about uncertainty by trial and error, such as hypothesis testing³¹ and sequence prediction³².

Different brain regions may contribute to the selection of behavior based on estimated probability under other circumstances. In a probabilistic classification task in which decisions were based on the relative accumulation of information toward one choice or another³³, activation of insular, lateral prefrontal and parietal cortices increased with increasing uncertainty, becoming maximal when there was equal evidence for each of two choices. This set of regions overlaps with those implicated in behavioral control and executive processing^34-37, suggesting that information about probability may be an important input to neural control systems. Posterior parietal cortex, in particular, may be critical for many sorts of judgments about probability, value, and derivatives such as expected value because of its contributions to calculation and estimation³⁸.

Converging evidence from primate electrophysiology and human neuroimaging has identified brain regions associated with utility. The receipt of a rewarding stimulus (‘outcome’ or ‘experienced’ utility) evokes activation of neurons in the ventral tegmental area of the midbrain, as well as in the projection targets of those neurons in the nucleus accumbens within the ventral striatum and in the ventromedial prefrontal cortex³⁹. Computational modeling of the response properties of dopamine neurons has led to the hypothesis that they track a reward prediction error reflecting deviations from expectation^40,41. Specifically, firing rate transiently increases in response to unpredicted rewards as well as to cues that predict future rewards, remains constant to fully predicted rewards and decreases transiently when an expected reward fails to occur. Similar results have been observed in human fMRI studies. For example, in a reaction-time game^42,43, activation of the ventral striatum depends on the magnitude of the expected reward but is independent of the probability with which that reward is received⁴³. A wide variety of different reward types modulate ventral striatum activation, from primary rewards such as juice^44,45 to more abstract rewards such as money^46,47, humor⁴⁸ and attractive images⁴⁹. Indeed, even information carried by unobtained rewards (a fictive error signal) modulates the ventral striatum⁵⁰.

An important topic for future research is whether representations of value and probability share, at least in part, a common mechanism. While subjects played a gambling task, activation of the ventral striatum showed both a rapid response associated with expected value (maximal when a cue indicated a 100% chance of winning) and a sustained response associated with uncertainty (maximal when a cue indicated a 50% chance of winning)⁵¹. These results demonstrate the potential complexity of uncertainty representations in the brain, such that a single psychological state may reflect multiple overlapping mechanisms.

Uncertainty influences on neural systems mediating choice

Recent work in neuroeconomics has examined the effects of uncertainty on the neural process of decision making. These studies involve trade-offs between economic parameters, such as a choice between one outcome with higher expected value and another with lower risk. Because of the complexity of these research tasks, such studies are typically done using fMRI in human subjects, with monetary rewards.

In studies of risky choice, an intriguing and common result is increased activation in insular cortex when individuals choose higher-risk outcomes over safer outcomes. In an important early study, subjects played a ‘double-or-nothing’ game⁵². If subjects chose the safe option, passing, they would keep their current winnings. If subjects instead chose to gamble, they had a chance of doubling their total, at the risk of losing it all. Activation in the right anterior insula increased when the subjects chose to gamble, and the magnitude of insular activation was greatest in those individuals who scored highest on psychometric measures of neuroticism and harm avoidance. Under some circumstances, avoidance of risk may be maladaptive. For example, in a financial decision-making task that involves choices between safe ‘bonds’ and risky ‘stocks’⁵³, when insular activation is relatively high before a decision, subjects tend to make risk-averse mistakes; that is, they chose the bonds, even though the stocks were an objectively superior choice. Insular activation is also robustly observed when decision-making impairments lead to increased risk^54,55. Insular activation may reflect that region’s putative role in representing somatic states that can be used to simulate the potential negative consequences of actions^56,57, as when people reject unfair offers in an economic game at substantial cost to themselves⁵⁸.

Individuals tend to avoid risky options that could result in either a potential loss or a potential gain, even when the option has a positive expected value. Most people will reject such gambles until the size of the potential gain becomes approximately twice as large as the size of the potential loss; this phenomenon is known as loss aversion. Loss aversion may reflect competition between distinct systems for losses and gains or unequal responses within a single system supporting both types of outcomes. Both gains and losses evoke activation in similar regions, including the striatum, midbrain, ventral prefrontal cortex and anterior cingulate cortex, with activation increasing with potential gain but decreasing with potential loss⁵⁹. Consistent with the assumptions of economic prospect theory⁷, activation in these regions is more sensitive to the magnitude of loss than that of gain. Whether losses and gains are encoded by the same system, as suggested by these results, or by more than one system^60,61 remains an open and important question. A potential resolution is suggested by work demonstrating that prediction errors for losses and for gains are encoded in distinct regions of the ventral striatum⁶².

Fewer studies have investigated the neural mechanisms recruited by uncertainty associated with ambiguity, or the lack of knowledge about outcome probabilities. In a study notable for its use of parallel neuroimaging and lesion methods⁶³, subjects chose between a sure reward ($3) and an outcome with unknown probability ($10 if a red card was drawn from a deck with unknown numbers of red and blue cards). On such ambiguity trials, most subjects preferred the sure outcomes. On the control trials, subjects were faced with similar trials that involved only risk (a sure $3 versus a 50% chance of winning $10). Thus, these two types of trials were matched on all factors except whether uncertainty was due to ambiguity or to risk. Ambiguity, relative to risk, increased fMRI activation in the lateral orbitofrontal cortex and the amygdala, whereas risk-related activation was stronger in the striatum and precuneus. In the same behavioral task, subjects with orbitofrontal damage were much less averse to ambiguity (and to risk) than were control subjects with temporal lobe deficits. One potential interpretation of these converging results is that aversive processes mediated by lateral orbitofrontal cortex, which is also implicated in processes associated with punishment⁶¹, exert a greater influence under conditions of ambiguity.

Similar comparisons of the neural correlates of risk and ambiguity have been done by other groups^64,65. An fMRI study found that subjects’ preferences for ambiguity correlate with activation in lateral prefrontal cortex, whereas preferences for risk correlate with activity in parietal cortex⁶⁴. These results link individual differences in economic preferences to activation of specific brain regions. However, there were no activation differences between ambiguity and risk in the ventral frontal cortex or in the amygdala. The lack of such effects may reflect the extensive training of the fMRI subjects, who were highly practiced at the experimental task and close to ambiguity-and risk-neutral in their choices. Thus, lateral prefrontal and parietal cortices may support computational demands of evaluating uncertain gambles, whereas orbitofrontal cortex and related regions may support emotional and motivational contributions to choice. None of these brain regions specifically processes ambiguity or risk; instead, each may contribute different aspects of information processing that are recruited to support decision making under different circumstances.

When faced with uncertainty, decision makers often try to gather information to improve future choices. Yet collecting information often requires forgoing more immediate rewards. This tension between seeking new information and choosing the best option, given what is already known, is called the ‘explore-exploit’ dilemma. In a study investigating the potential neural basis for such trade-offs⁶⁶, subjects chose between four virtual slot machines, each with a different, unknown, and changing payoff structure. When subjects received large rewards from their chosen machine, there was increased activation in the ventromedial prefrontal cortex, with deviations from an expected reward level represented in the amplitude of ventral striatal activation. Most intriguingly, those trials in which subjects showed the most exploratory behavior were associated with increased activation in frontopolar cortex and the intraparietal sulcus. An important topic for future research will be identifying how these latter regions (and, presumably, others) modulate reward processing and behavioral control.

Neuronal correlates of outcome uncertainty and risky decision making

Neuroimaging studies have thus implicated several brain regions in uncertainty-sensitive decision making. Neurophysiological studies in animals confronted with reward uncertainty and risky decisions have only begun to explore the computations made by neurons in these areas and others. As described above, dopamine neurons fire a phasic burst of action potentials after the delivery of an unexpected reward as well as after the presentation of cues that predict rewards. This same system may contribute to the evaluation of reward uncertainty as well. When monkeys are presented with cues that probabilistically predict rewards, dopaminergic midbrain neurons respond with a tonic increase in activity after cue presentation that reflects reward uncertainty (Fig. 2a)⁶⁷. Neuronal activity peaks for cues that predict rewards with 50% likelihood and decline as rewards become more or less likely. These results thus suggest that dopamine neurons may convey information about reward uncertainty. However, neuronal activity correlated with uncertain rewards may instead reflect, or even contribute to, the subjective utility of risky options, which was not measured in that study⁶⁷, or the trial-wise back-propagation of reward prediction errors⁶⁸ generated by dopamine neurons during learning (but see ref. 69). Nevertheless, the observation that lesions of the nucleus accumbens, a principal target of midbrain dopamine neurons, enhance risk aversion in rats⁷⁰, as well as the increased incidence of compulsive gambling in patients with Parkinson’s disease taking dopamine agonists^71,72, provides some functional support for dopaminergic involvement in risky decision making.

Figure 2.

Neuronal correlates of risky rewards. (a) Midbrain dopamine neurons in monkeys increase firing in anticipation of probabilistically delivered juice rewards (after C.D. Fiorillo et al., 2003)⁶⁹. (b) Neurons in posterior cingulate cortex preferentially signal uncertain rewards in a visual gambling task. RF, receptive field target. (c) Changes in neuronal activity following a change in the identity of the uncertain target mirrored the development of preferences for that target (b,c after A.N. McCoy and M.L. Platt, 2005)⁷⁵.

Regardless of which brain system provides initial signals about the uncertainty of impending rewards, an important question for neurobiologists is how such information about risk influences preferences and how these preferences are mapped onto the actions that express decisions. Neurons in posterior cingulate cortex (CGp) may be involved in risky decision making⁷³. Based on anatomy, CGp is well situated to translate subjective valuation signals into choice because it makes connections with brain areas implicated in processing reward, attention and action⁷⁴. Moreover, this area is activated during decision making when rewards are uncertain in either amount⁷⁵ or time⁷⁶, and the magnitude of activation depends on the subjective appeal of proffered rewards⁷⁷. Finally, neurophysiology shows that CGp neurons respond to salient visual stimuli⁷⁸, after visual orienting movements^78,79 and after rewards⁸⁰, and that all these responses scale with reward size and predictability⁸⁰. Together, these data suggest that CGp has an evaluative role in guiding behavior^79,80.

In a gambling task to assess whether reward-related modulation of neuronal activity in CGp reflects subjective utility or the objective properties of available rewards⁷³, monkeys were given a choice between two options on a computer monitor that were matched for their expected value. Choosing the safe option always resulted in a medium-sized squirt of juice. Choosing the risky option resulted in a 50% chance of a large squirt of juice and a 50% chance of a small squirt of juice. In this task, monkeys favored the risky option, even in a second experiment when it paid less, on average, than the safe one. Neurons in CGp closely mirrored this behavioral bias. The studied neurons responded more strongly after risky choices (Fig. 2b), and these responses correlated with monkeys’ overall preference for the risky option rather than with the option’s objective value (Fig. 2c). These data are consistent with the hypothesis that CGp contributes to decision making by evaluating external events and actions with respect to subjective psychological state. One concern might be whether modulation of neuronal activity in CGp associated with choosing risky options reflects arousal, which might be elevated when making a risky choice⁸¹. Heart-rate, a somatic correlate of physiological arousal, did not vary between high-risk and low-risk blocks of trials, thus indicating that elevated arousal cannot completely account for these results. However, the responses of CGp neurons to uncertain gambles may reflect reward uncertainty per se rather than subjective preference for the risky option, as these variables were not dissociated in that study. Further studies that systematically dissociate risk and preference will be needed to address these questions.

CONCLUSIONS

Given the pervasiveness of uncertainty, it is hardly surprising that research from a wide range of disciplines—psychology, neuroscience, psychiatry and finance, among many—has already addressed some key questions: How should investors deal with financial risk, and how do they actually deal with it? What brain systems estimate outcome uncertainty? What is different about those systems in people who make bad choices? These are complex but tractable questions, and scientists have made significant progress toward their solution. We now understand that uncertainty strongly biases choice, that these biases vary across individuals and that specific brain systems contribute to biased decision making under uncertainty. These findings represent real advances, the importance of which should not be minimized.

Yet deep questions remain unanswered and even unaddressed. We contend that fundamental advances in the study of decision making will only arise from consilient integration of expertise and techniques across traditionally independent fields. Neuroscience data can constrain behavioral studies; individual differences in genetic biomarkers can inform economic models; developmental changes in both behavior and brain can guide studies of decision making in adults. Only through explicit interdisciplinary, multimethodological and theoretically integrative research will the current plethora of perspectives coalesce into a single descriptive, predictive theory of risk-sensitive decision making under uncertainty.