Three Studies on the Neuroeconomics of Decision-Making when Payoffs are Real and Negative

Abstract

Purpose

We summarize three previous neuroeconomic studies with two features that distinguish them from most others in experimental economics: (1) the use of physical pain to induce incentives and (2) acquisition of data on brain activation levels. By correlating behavior when payoffs are painful with brain activation, we are able to test for the neurobiological relevance of important phenomena previously observed in experimental studies that are at odds with classical economic theories of decision-making. These specific phenomena are (a) negative discounting of future payoffs; (b) nonlinear probability weighting; (c) the experience of regret and rejoice when making a decision under risk.

Methodology/approach

The expectation of pain is created through the use of mild electric shocks to the top of the foot. Pain confers disutility, so decisions are made in the domain of losses relative to the status quo. Simultaneous with these decisions, brain activation data is acquired through functional magnetic resonance imaging (fMRI).

Findings

We find evidence for negative time discounting of electric shocks. Participants who exhibited the most extreme forms of this discounting were distinguished by early and robust activation of a subset of the cortical pain matrix. We also find evidence for probability weighting in the domain of electric shocks, which is manifest at the neural level. We find evidence both behaviorally and neurally for regret and rejoice functions for painful outcomes.

Originality/value of chapter

Previous experimental economic studies in the domain of losses have typically used monetary rewards. Here, we report behavioral effects and neural correlates using pain.

1. Introduction

In this chapter, we summarize three neuroeconomic studies previously reported in (Berns et al., 2006; Berns, Capra, Moore, & Noussair, 2007; Berns, Capra, Chappelow, Moore, & Noussair, 2008) and Chandresakhar, Capra, Moore, Noussair, and Berns (2008). There are two features that distinguish these three studies from most others in the experimental economic literature. The first feature is that in these studies we use physical pain to induce incentives. Pain is created by applying mildly painful electric shocks to participants. In contrast, almost all other experimental economic studies use monetary incentives, either real or hypothetical. The second feature is that in addition to recording decisions, we acquire data on brain activation levels. By correlating behavior when payoffs are painful and real with brain activation, we are able to test for the neurobiological relevance of important phenomena previously observed in experimental studies that are at odds with classical economic theories of decision-making. These specific phenomena are (a) negative discounting of future payoffs; (b) nonlinear probability weighting; and (c) the experience of regret and rejoice when making a decision under risk.

In the research described here, participants made decisions in the domain of losses relative to the status quo. This is a feature of many situations of interest in economics, but it is difficult to study in control environments. Consider, for example, a patient who is diagnosed with a serious health problem and must decide between alternative risky treatments. The best possible outcome under any treatment is the restoration of a status quo level of health. All other outcomes involve unfavorable results, such as failure of the treatment, emergence of uncomfortable side effects, or protracted morbidity. Another example is a defendant in a criminal court case faced with the choice of accepting a plea bargain or going to trial. The plea bargain offers a fixed jail sentence, whereas the trial is a lottery in which the best possible outcome is avoidance of prison. Note that in these two examples, the decisions are made over non-monetary variables, health level, and prison time. In contrast, previous experimental economic studies of decisions in the domain of losses have typically used monetary reward media. In the research that we report here, all outcomes are also in terms of a non-monetary medium, electric shock voltage.

It is reasonable to argue that using electric shocks to induce incentives results in decisions that are exclusively in the domain of losses, since the only possible outcomes are the receipt of a shock or the avoidance of a shock. The reason is that unless special experimental procedures are employed to habituate participants to receiving electric shocks, the status quo can be assumed to be the absence of a shock. In addition to providing a means of inducing non-monetary incentives, the use of electric shocks allows us to mitigate a methodological problem that arises in the study of decisions in the domain of losses. Because of the difficulty of inducing voluntary participation in studies where individuals can lose money, monetary losses have been studied with two techniques intended to circumvent the problem. The first technique is to have individuals make hypothetical decisions. The drawback of this technique is that it may fail to induce incentives to make decisions in accordance with one's true preferences, since there are no direct payoff consequences that result from the decisions. The second technique is to give individuals an initial endowment of cash at the beginning of or before the experimental session, from which losses incurred over the course of a session are subtracted. This second procedure, while creating incentives to make meaningful decisions, also has a drawback. Using this procedure, it is unclear whether decisions can be truly considered as taking place in the domain of losses relative to the status quo. While observed decisions under such a protocol do differ from those obtained when individuals are accumulating positive earnings over the course of an experimental session, it is unknown whether they reflect the same principles of decision-making as are present outside the laboratory when decisions involve losses relative to the status quo.

Physical pain satisfies non-satiation and dominance, the two precepts proposed by Smith (1982) concerning the appropriateness of the medium whereby incentives are induced in an experiment. Non-satiation is the property that the utility of a participant is monotonic in the level of the medium received. This allows an experimental researcher to have confidence that individuals are optimizing the value of the intended objective function by maximizing or minimizing the amount of the reward medium they receive. Non-satiation is satisfied in the studies described here since higher voltage or longer shocks yield more disutility than lower voltage and shorter shocks (indeed, the results of participant self reports confirm this). Dominance means that the difference in the quantity of the incentive medium received between alternative decisions outweighs the decision cost associated with trying to achieve higher or lower levels of the medium. For the studies described in this chapter, dominance is satisfied if the electric shocks are administered in large enough voltage gradations that the incentive to minimize the voltage received exceeds the cost of the decisions required. As will be seen in the three studies, it can be comfortably assumed that dominance is satisfied.

Physical pain also has several other properties that confer advantages over the use of cash payments for certain types of experiments. One feature is that pain is consumed immediately. This is typically not the case with money, since money may only be spent (i.e., translated into consumption) after the experimental session is over. When control over the timing of consumption is important, cash payments may not be appropriate. For example, it is not clear to us how the negative discounting experiment described in section two could be performed with monetary incentives. Another advantage of pain is that, unlike money, it is not transferable to other individuals, and thus must be consumed by the participant in the experiment himself. Finally, pain is of independent interest due to the fact that it is a consideration in many actual decisions that individuals face, such as the health levels and jail times examples explained earlier.

The first study focuses on the nature of negative time discounting. Negative discounting (see Lowenstein, 1987, or Caplin & Leahy, 2001) is a formulation of the intuition that an individual at times may wish to bring forward an adverse experience to “get it over with,” or to postpone a favorable experience as if they wish to “savor” it. The study described in Section 2, also reported in Berns et al. (2006), considers the motivation behind the negative discounting of losses, with a focus on the existence and the nature of dread, a disutility associated with the anticipation of a future negative event. Traditional experimental economic methods, which record decisions, can accumulate evidence consistent with the existence of dread by showing that individuals try to avoid it. However, there are alternative explanations for decision patterns, which are consistent with negative discounting but exclude dread. One of these alternative explanations is that the disutility of the outcome itself increases with the length of time an individual waited for it (Kohno et al., 2003), and an individual seeks to bring the aversive outcome forward in time to lower the disutility when it is realized. That is, rather than focusing on the anticipation of the outcome, it focuses on the outcome itself.

Because fMRI technology provides information about process, it allows researchers to establish whether there is really an aversive experience associated with dread. In addition, it is possible to study the time profile of the aversive experience as the unpleasant outcome approaches. Characterization of the time profile would permit the researcher to distinguish between the two explanations of negative discounting described earlier. The use of painful shocks is particularly well suited to studying the existence of dread. Indeed, dread is defined only over outcomes that yield negative payoffs relative to the status quo, and so negative payoffs must be induced. In addition, the fact that in this particular experiment the researcher must exercise precise control over the timing of consumption of the incentive medium means that monetary payments cannot be used, but electric shocks can be an effective reward medium.

The second study, presented in Section 3 and also reported in Berns et al., (2007, 2008), concerns probability weighting. This is the notion that individuals make decisions as if they transform probabilities by a function π(p), where p is the probability of an event. Probability weighting is a well-documented empirical phenomenon (see Starmer, 2000, for a survey) and is one of the core assumptions of prospect theory (Kahneman & Tversky, 1979). This contrasts with expected utility theory, which assumes that individuals choose between lotteries over uncertain outcomes using the objective probabilities of each outcome as the weight they place on each potential outcome. Prospect theory assumes that decisions are consistent with maximization of the average utility of outcomes, but with a transformation of the probabilities of each outcome used to calculate the average. Most of the empirical evidence suggests that π(p) has an inverted S form, involving the overweighting of moderately small probabilities and the underweighting of relatively large ones. In the domain of losses, this pattern of inverted S-shaped probability weighting has only been established in experimental studies with hypothetical monetary lotteries, and with real losses that are subtracted from a fixed endowment of money. The use of electric shocks as a stimulus permits the realization of outcomes that are clearly inferior to the status quo. The use of neuroeconomic methods allows one to consider whether probability weighting originates at the stage of decision-making or whether it occurs in the absence of choice. This is because fMRI technology registers the response to the display of the lottery, in the absence of any decision.

The third experiment that we review concerns the nature of regret and rejoice (or rejoicing), and will be referred to as the “regret experiment” to facilitate exposition. Regret occurs when a choice an individual makes yields a payoff that is lower than an alternative choice would have yielded. Rejoicing occurs when an outcome of an individual's decision yields a higher payoff than would have resulted from an alternative decision. Regret and rejoice are ex-post phenomena experienced after a decision is made. This means that in decision-making experiments, their existence is inferred indirectly. While regret can be created in the domain of positive payoffs with the inducement of opportunity costs of foregone rewards, it is of special interest to study regret in the domain of negative payoffs. One reason for this is that making a decision that results in an outcome worse than the status quo typically results in real costs rather than opportunity costs. Thus, there may be more scope for regret – or for rejoicing at avoiding an adverse outcome – to exert a powerful effect when the decision is taken in the domain of losses. The use of neuroeconomic methods allows us, at least in principle, to identify whether the experience of an outcome is different when there is reason to believe that regret and rejoicing are occurring, and whether what is experienced in a situation where regret (rejoicing) is present is unpleasant (pleasant).

2. Why Do People Want to Get an Adverse Experience Over with as Soon as Possible?

The experiment described in this section (see Berns et al., 2006 for more detail) was designed to distinguish between two potential explanations for why observed decisions might be consistent with negative discounting. One possibility is that, in addition to the utility of the outcome, the anticipation of the outcome may yield utility in itself. For impending negative outcomes, this means that an experience would be felt of “dread,” a disutility incurred while waiting for the outcome. This disutility can be reduced if the time of the outcome is brought forward. In this account, waiting for an adverse experience is unpleasant for an individual, so it is optimal to get the experience behind him. A second possibility is that waiting for the outcome increases the disutility of the outcome itself. A bad outcome “hurts” more when it arrives because one waited longer for it. This would also account for a willingness to bring adverse outcomes forward in time, if possible. In the experiment described here, these two explanations imply a different time course of activation in the pain matrix, a network of brain regions associated with unpleasant or noxious stimuli (Ploghaus et al., 1999; Koyama, McHaffie, Laurienti, & Coghill, 2005; Rajj et al., 2005; Tracy, 2001; Craig, 2003; Peyron et al., 2000). Measuring the time profile of activation allows us to distinguish between the two explanations.

The pain matrix includes the following regions: the primary somatosensory cortex (SI), the secondary somatosensory cortex (SII), and the posterior insula, all of which are associated with the somatosensory experience of pain. Activity in the anterior insula, the rostral ACC, and the amygdale, has been linked to the visceral and emotions aspects of pain. Preparation for a withdrawal response has been linked to activity in mid-ACC and supplementary motor area (SMA). We take measures of activation of these regions as a measure of disutility, in an analogous manner to authors who have interpreted activation of other regions as a measure of positive utility (Camerer et al., 2005; Glimcher, 2002; Hsu, Bhatt, Adolphs, Tranel, & Camerer, 2005; Huettel, Stowe, Gordon, Warner, & Platt, 2006; Knutson, Taylor, Kaufman, Peterson, & Glover, 2005; Preuschoff, Bossaerts, & Quartz, 2006).

The experiment consisted of two phases, and each phase was divided into a sequence of trials. In each trial, cutaneous electrical shocks were delivered using a Grass SD-9 stimulator (West Warwick, RI) through shielded, gold electrodes placed 2–4 cm apart on the dorsum of the left foot. Each shock was a monophasic pulse of 10–20 ms duration. The Grass stimulator was modified by attaching a servo-controlled motor to the voltage potentiometer. The motor allowed for computer control of the voltage level without compromising the safety of the electrical isolation in the stimulator. The motor was controlled by a laptop through a serial interface. During the experiment, individuals were lying down in an fMRI scanner, and their skin conductance was registered. Scanning was conducted with a Siemens 3T Trio wholebody scanner. After acquisition of a high-resolution T1-weighted scan, fMRI of the BOLD response was performed (TR = 2350ms, TE = 30 ms, 64 × 64 matrix, 35 axial slices, 3 mm cubic voxels). To prevent electrical artifacts in the fMRI signal due to shock deliveries, the latter occurred during a 50 ms pause after each volume, yielding an effective TR = 2,400 ms.

Prior to scanning, the voltage range was titrated for each participant. The detection threshold was determined by delivering pulses starting at zero volts and increasing the voltage until the individual indicated that he could feel them. This minimum perception threshold is denoted here as V_min. The voltage was increased further, while each participant was instructed, “When you feel that you absolutely cannot bear any stronger shock, let us know – this will be set as your maximum; we will not use this value for the experiment, but rather to establish a scale. You will never receive a shock of maximum value.” This maximum is denoted as V_max. The purpose of this procedure was to control for the heterogeneity of the skin resistance among subjects and to administer potentially painful stimuli in an ethical manner. Admittedly, this is an imperfect procedure in that some participants may not have indicated their maximum value, introducing another source of heterogeneity between subjects (willingness to reveal their maximum). We index the strength of the shock administered to an individual by s, where the associated voltage for an individual is V_s = s * V_max + (1 s) * V_min. For the experiment, s took on values of 10%, 30%, 60%, and 90%. The stimuli in the experiment were of the form (s_t, d_t) where s_t is the shock strength at time t, and d_t is the delay between when the stimulus is shown and when the shock occurs.

In the first phase of the experiment each of 32 individuals participated in the 96 trials that constituted the Passive Phase of the experiment. Each trial began with the presentation of a cue that indicated both the voltage level and the amount of time one would have to wait for the outcome. The displays and timing are illustrated in Fig. 1. Shocks were delivered to the dorsum of the left foot on a 100% reinforcement schedule. After this was completed, the Active Phase of the experiment took place. The neural activation patterns were registered as the voltage delay combinations were displayed, and as individuals waited for the shock. After the realization of the outcome, the subject was required to rate the experience of the trial, in a range between “very unpleasant” and “very pleasant.” To indicate his rating, he marked a location on a visual analog scale (Noussair, Robin, & Ruffieux, 2004) using a cursor operated with his hand control.

Fig. 1.

Timing and Displays of a Trial, Dread Experiment.

After the Passive Phase ended, the Active Phase took place. In this phase, participants were presented in a series of trials with pairs of voltage and delay: for example, a 90% voltage shock in 3 sec and a 60 percnt shock in 27 sec. Individuals were required to choose which of the two options they would prefer to receive. The shock was then administered in accordance with the choice of the participant. Choosing the shorter delay could not speed up the experiment, as each trial lasted the length of the longer of the two choices (when the shorter duration was chosen, the extra time was added to the interval between the shock and the next trial).

In the Active Phase, in trials when the voltages were identical and the delays differed, 78.9% of the choices were in favor of the shorter delay, which is consistent with negative discounting. 27 of the 32 individuals chose the shorter delay in a majority of trials. Some individuals were even willing to accept a larger shock if it were administered sooner. The participants were classified into 23 mild and nine extreme dreaders. The extreme dreaders were those individuals who preferred more voltage sooner to less voltage later, and the mild dreaders were those who dreaded only to the extent of shortening the delay at a given voltage but were not willing to take more voltage just to get the shock over with. These classifications were consistent with individuals' self reports. Extreme dreaders reported a worse experience when they had to wait longer for the shock.

We examined the fMRI response to the shock itself and identified brain regions sensitive to shock amplitude by a linearly increasing contrast across voltage levels. We then selected 12 subregions of this map that intersected the pain matrix. A voltage-weighted contrast on the instantaneous response to the shock revealed a map consistent with previous reports of the pain matrix. Although a significant effect of the length of delay was observed in the right SII, the predominant pattern in the pain matrix was that waiting did not change the response to the shock itself; also, there was not a differential voltage sensitivity between mild and extreme dreaders. Therefore, whatever differentiated the two groups must have occurred during the waiting period, and it does not appear that the preference for expediting negative outcomes results from any impact of waiting on the utility of the outcome itself. Fig. 2 illustrates the activation as a function of voltage and delay for several regions for the mild and extreme dreaders separately.

Fig. 2.

Activation of Selected Brain Regions as a Function of Voltage and Delay, Dread Experiment.

To study differences between mild and extreme dreaders during the waiting period, we performed a time-series analysis on the regions of interest (ROIs). We used Loewenstein's (1987) model for the utility of anticipation to test the hypothesis that the distinguishing characteristic between mild and extreme dreaders lies in the prospective response to future outcomes. In this model, the present value of a delayed act of consumption is divided into two components: (a) the utility from consumption and (b) the utility from anticipation (dread). Assuming instantaneous consumption at the time (T) of shock delivery, the present value at time (t) of a future act of consumption is the utility of consumption U discounted by an exponential function with rate r = Ue^{−r (T − t)}. In addition to the discounted consumption utility, anticipation (dread) also yields utility. For simplicity, we assumed that the instantaneous intensity of dread was constant and that the present value was this constant, α, multiplied by the time remaining until the shock. The constant α is referred to as the dread factor. Thus, combining the terms for dread and discounted consumption, the present value of future utility at time t, U(V,t) = U(V) * [a(T−t) + e^{−r (T − t)}], where U(V) is the utility of the shock with voltage V occurring at time T, α is the dread factor, and r is the discount rate. Differences in the utility of dread are reflected as differences in the dread factor α.

We observed that all of the contralateral (right hemisphere) regions of interest and the caudal ACC displayed time courses with dread factors significantly different from zero, but this was an effect observed primarily in the extreme dreaders and not the mild dreaders. Both SI and SII showed marked elevations in activity after the presentation of the cue – an elevation which continued to increase in advance of the shock. But the initial elevation in SI, SII, and right posterior insula, which was measured by the dread factor, was significantly greater in the extreme dreaders. The time course in the caudal ACC displayed a significant dread factor for only the extreme dreaders. The right amygdala had a significant dread factor for both groups, but was not significantly different between mild and extreme dreaders. From the time course of the response in these regions, coupled with its predominance in individuals who showed the most extreme behavioral evidence of not wanting to wait, we conclude that the component of anticipation that can be specifically attributed to dread appears in the posterior elements of the cortical pain matrix (SI, SII, the posterior insula, and the caudal ACC) and not the anterior ones (the anterior insula and the rostral ACC).

Both SI and SII have generally been associated with the physical intensity of noxious stimulation (Tracy, 1999; Craig, 2003; Petrovic, Petersson, Hansson, & Ingvar, 2002; Bentley et al., 2004), whereas the caudal ACC has been associated with the attentive component of pain (Vogt, 2005; Bentley, Derbyshire, Youell, & Jones, 2003). The localization of dread to the posterior elements of the matrix suggests that dread has a substantial attentive component. Both the mild and extreme dreaders displayed time courses of activity in SI, SII, the caudal ACC, and the posterior insula that were consistent with the utility-based theory of dread. The more anterior, emotional components of the pain matrix (e.g., the anterior insula, the rostral ACC, and the amygdala) did not have such time courses, indicating that dread is not primarily fear of the outcome. Moreover, it was the significantly different dread factor in the posterior divisions that most clearly distinguished mild from extreme dreading behavior when individuals subsequently had to make decisions regarding wait times. With regard to nociceptive inputs, both SI and SII receive afferent signals from the posterior portion of the ventromedial nucleus of the thalamus, whereas the ACC receives input from the mediodorsal nucleus (Craig, 2003). As the terminal fields from the spinothalamic system, these regions naturally show activations that track stimulation voltage. The amygdala, whose role in aversive conditioning is well known (Phelps, 2006), displayed a significant dread response on the right side, but this was not significantly different between the mild and extreme dreaders.

3. Probability Weighting

A similar methodology to the one described earlier was used to consider the issue of probability weighting. In this second study, a total of 37 (20 female, 17 male) people were scanned using fMRI (28 were used for the analysis due to signal artifacts for nine subjects). The general framework and voltage setting procedure were similar to the experiment described in Section 2, except that the shocks were delivered probabilistically and all shocks occurred with the same delay. The study is reported in more detail in Berns et al. (2007) and in Berns et al. (2008).

After the voltage titration, the experiment began, and was divided into two phases. The first phase of the experiment, again called the Passive Phase, consisted of 120 trials. At the beginning of each trial, each participant was presented with a pie chart that conveyed both the voltage of the impending shock and the probability with which it would be received. The display thus defined a lottery consisting of a shock with voltage level, s, and a probability, p, in trial t, (s^t, p^t). The size of the pie chart indicated the strength of the shock that might be applied in the current trial, with the area of the pie chart equaling s times the area of the outer reference circle (denoting V_max). The percentage of the inner circle that was filled in red indicated the probability with which the shock was to be administered. The possible probabilities were 1/6, 1/3, 2/3, 5/6, and 1. With the four voltage levels, this yielded 20 voltage-probability combinations, each of which was presented six times during the 120 trials that made up the passive phase of a session.

In the second phase, the Active Phase, each individual faced a sequence of 60-pairwise choices from the set of probability/shock combinations presented in the passive phase. In each round, two lotteries appeared side by side, and subjects were required to choose one of them, using the keypad provided to them. The experimenter chose the pairs so that in every trial one alternative always specified both a higher voltage shock as well as lower probability than the other alternative. In other words, if ${\mathit{\text{s}}}_A^t$ and ${\mathit{\text{s}}}_B^t$ are the strengths of the shocks that may be applied under choices A and B respectively in trial t, it was always the case that ${\mathit{\text{s}}}_A^t>{\mathit{\text{s}}}_B^t\leftrightarrow {\mathit{\text{p}}}_A^t<{\mathit{\text{p}}}_B^t$, where ${\mathit{\text{p}}}_A^t$ is the probability that a shock is applied under choice A in trial t. Subjects received a shock with the voltage specified by their choice and with the indicated probability.

In the passive phase, a well-defined network of brain regions was active in response to the cue, and the pattern of activation could be largely dissociated into magnitude-sensitive and probability-sensitive regions. The probability-sensitive regions are given in Table 1. Some of these regions were clearly related to the low-level processing of visual stimuli (e.g., visual cortex). Other regions, however, encoded aspects of the anticipated voltage and/or probability. The probability of receiving a shock was most significantly correlated with activity in the bilateral inferior parietal cortex, near the temporal–parietal–occipital junction, whereas the magnitude of the impending shock was correlated with bilateral activity in the insula/superior temporal cortex, precuneus, cerebellum, and a region of the precentral gyrus associated with the foot. Many other regions, including the caudate/subgenual ACC, displayed negative correlations with the impending magnitude of the outcome or the probability of receiving a shock. There was very little anatomical overlap between the magnitude and probability maps, with the exception of the ACC and supplementary motor area dorsal to it, in which interaction of the two dimensions was observed.

Table 1.

Probability Sensitive Regions, Probability Weighting Experiment.

Region	MNI Coordinates	T-Statistic	NPRR(l/6) > 1/6?
Probability sensitive regions (Positive correlation)
R Cingulate Gyrus (BA 24)	9, −18, 39	5.88	N
L Inferior Parietal Gyrus (BA 40)	−57, −57, 42	4.98	N
R Superior Temporal Gyrus (BA 22)	63, −48, 12	4.97	Y
R Superior Frontal Gyrus (BA 6)	9, 3, 63	4.92	Y
R Inferior Parietal Gyrus (BA 40)	66, −36, 30	4.91	N
L Cingulate Gyrus (BA 24)	−6, 0, 45	4.89	N
L Postcentral Gyrus (BA 7)	−9, −57, 69	4.64	Y
Probability sensitive regions (Negative correlation)
L Visual Cortex (BA 18)	−30, −96, 6	5.58	N
L Posterior Cingulate (BA 29)	−12, −51, 6	4.66	Y
R Lingual Gyrus (BA 19)	18, −57, −3	4.30	N
L Anterior Cingulate (BA 24)	−6, 21, −3	4.05	Y
R Thalamus	21, −30, −3	4.03	Y
L Middle Temporal Gyrus (BA 37)	−48, −60, −3	4.00	Y

The ACC has been previously implicated in modulating decision weights for risky financial decisions (Paulus & Frank, 2006; Knutson et al., 2005). However, unlike Paulus and Frank, we found no significant correlation between individuals' levels of ACC activation and the curvature of their probability weighting implied by their lottery choice decisions. One possible reason is that the activation we measured represented a passive response in the absence of a decision. The ACC is a prime candidate for the integration of magnitude and probability information – even in the absence of a required response, because of its role in integrating the affective system and the motor response system (Botvinick, Braver, Barch, Cohen, & Carter, 2001; Critchley, Mathias, & Dolan, 2001; Miller & Cohen, 2001). Although prospect theory assumes functional separability of probability and utility functions, the multiplication of these two dimensions must occur somewhere. The ACC is an integral component of the cortical pain matrix, and current evidence suggests that the ACC integrates several dimensions of the subjective pain experience, including emotional and attentional components (Craig, 2003; Ploghaus et al., 1999; Vogt, 2005).

In order to determine a probability weighting function with fMRI measurements, we define a neurobiological probability response ratio (NPRR). This is the ratio of (i) the activation level associated with a lottery with two possible outcomes, a non-null outcome denoted by x, and a null outcome, and (ii) the non-null outcome with certainty. The activation associated with the null outcome, the non-null outcome, and the lottery are denoted by y₀, y(x,1), and y(x,p) respectively. The NPRR is given by:

$$\text{NPRR}(x,p)=\frac{\left[y(x,p)-y_0\right]}{\left[y(x,1)-y_0\right]}$$

We choose y₀ as the activation associated with p = 1/3 for our study in light of an extensive behavioral literature on probability weighting in the financial domain suggesting that the probability weighting function crosses the diagonal in the vicinity of p = 0.4 (Abdellaoui, 2000; Kahneman & Tversky, 1979; Tversky & Kahneman, 1992; Wu & Gonzalez, 1996). In our experiment, the probability prospect that came closest to meeting these requirements was p = 1/3. Thus, for purposes of our analysis we employ:

$$\text{NPRR}(x,p)=\frac{2}{3}\frac{\left[y(x,p)-y(x,1/3)\right]}{\left[y(x,1)-y(x,1/3)\right]}+\frac{1}{3}$$

Although this formula can be applied to all probabilities in the interval [0,1], its validity for probabilities less than 1/6 was untested in this experiment. We note, for example, that p = 0 may be a case that is treated categorically different by the brain. Because of deviations from normality in the distributions of NPRR, we used the median of the means of the three highest voltages value at each probability to estimate the central tendency. 95% confidence intervals of the median were computed according to this formula: [(N+1)/2] ± 1.96 * (√N)/2, which corresponded to subjects 9 and 20 in an ordered listing of the 28 participants. To test the null hypothesis of linear probability weighting, and thus the expected utility hypothesis, we used the 95% confidence intervals at p = 1/6, 2/3, and 5/6. Because previous behavioral evidence suggests non-linear probability weighting has an inverted S-shaped functional form, overweighting low-probability and underweighting high-probability events, we were particularly interested in whether NPRR(l/6) > 1/6, NPRR(2/3)<2/3, and NPRR(5/6)<5/6.

Our fMRI data, however, are consistent with a biological bias toward non-linear weighting of probabilities. Fig. 3 illustrates the activation of several of the regions of interest as a function of probability. In most cases, including in all of the regions shown in the figure, the NPRR displayed an inverted-S shape, which is characteristic of the behavioral probability weighting function hypothesized in prospect theory. As shown in the last column of Table 1, the majority of regions have the property that NPRR(l/6) is significantly different from (greater than) 1/6. The majority of regions also have the properties that NPRR(2/3)<2/3, and NPRR(5/6) > 5/6 (results are available from the authors). Although we did not find significant departures from linearity for each probability value for each brain region, most regions displayed a significant departure from linearity for one of the probabilities. Importantly, when such a departure from linearity was observed, it was always in the direction predicted by prospect theory.

Fig. 3.

Activation of Selected Brain Regions as a Function of Probability, Probability Weighting Experiment.

Regions that have been previously associated with probabilistic decisionmaking, notably in the parietal cortex (Huettel et al., 2006; Platt & Glimcher, 1999; Shadlen & Newsome, 2001; Glimcher, Dorris, & Bayer, 2005), showed significant forms of nonlinearity. We found maps for both probability and magnitude adjacent to each other near the temporo-parietal junction. The temporo-parietal junction, including both the inferior parietal lobule and superior temporal gyrus, has been implicated in at least one previous study of risky financial decision-making (Paulus & Frank, 2006). This region has also been conjectured to play a critical role in the judgment of true and false beliefs originating from other people (Grezes, Frith, & Passingham, 2004; Saxe & Kanwisher, 2003; Sommer et al., 2007) as well as attention shifting (Shulman, Astafiev, McAvoy, d'Avossa, & Corbetta, 2007). Its role in our experiment may operate similarly, if more generally outside the specific circumstance of decision-making: judging the likelihood of receiving and avoiding an aversive outcome (Mitchell, 2007).

Another region that showed non-linear weighting of probabilities was a large bilateral cluster encompassing the anterior insula and the superior temporal gyrus. The anterior insula, in particular, has been previously associated with anticipatory responses to painful stimuli (Craig, 2003; Koyama et al., 2005; Ploghaus et al., 1999; Tracey, 2005). Although this region was identified by its monotonically increasing response to expected magnitude, it also showed a significantly non-linear NPRR, suggesting that the distortion of probabilities becomes intertwined with representations of the likelihood of subjective states. Notably, we did not find significant activations of the amygdala, which has previously been associated with learning associations between cues and shocks. However, this could be due to the extended nature of the cues we used, or to the tendency of the amygdala to be activated transiently during cue acquisition (LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998).

To test whether participants' decisions are consistent with non-linear probability weighting (see Berns et al., 2007, for more detail), we estimated values for γ in a standard specification for non-linear weighting (Tversky & Kahneman, 1992):

$$V(L)=\sum _i-\lambda \frac{p_i^{\gamma }}{{\left(\text{∑}_i{p}_i^{\gamma }\right)}^{1/\gamma }}|x_i{|}^{\alpha }$$

Across all 37 subjects, the estimated median value was 0.685 for γ and was thus consistent with an inverted S-shaped probability weighting function, and with previous studies. We also considered the incidence of common-ratio-violations. In our experiment, common-ratio-violations were observed when the lottery (V_h, p₁ = 1/6) was chosen over (V₁, p_h = 2/6), but (V₁, p_h = 4/6) was chosen over (V_h, p₁ = 2/6), or vice versa. Out of six possible instances in which each individual could commit a common-ratio-violation, the average number of violations was 1.95, which was significantly different from zero. In general, the directions of the violations were consistent with fanning out of indifference curves.

The main conclusion of the experiment is the existence of a consistent form of non-linearity observed in neurobiological probability response ratios that parallels the type of nonlinearity observed behaviorally. When observing choice behavior alone, the source of this nonlinearity is unknowable. Individuals, for example, might distort probabilities only when they make a decision. Our data, however, suggests that these distortions occur even in the absence of choice and thus are a property of a more fundamental process of how the brain transforms representations of probabilities into biological responses.

4. Regret and Rejoicing

In regret theory, regret and rejoicing affect the value of a lottery (Bell, 1982, 1983; Gilovich, Medvec, & Kahneman, 1998; Loomes & Sugden, 1982, 1987), and influence decisions between risky lotteries. A number of experimental studies have found that regret does influence decision-making (Bleichrodt, Cillo, & Diecidue, 2007; Cooke, Meyvis, & Schwartz, 2001; Janis & Mann, 1977; Sorum et al., 2004; Wolfson & Briggs, 2002; Zeelenberg, 1999; Zeelenberg, Beattie, van der Pligt, & de Vries, 1996) and can explain various economic phenomena (Smith, 1996; Tsiros & Mittal, 2000; Braun & Muermann, 2004; Dodonova & Khoroshilov, 2005; Muermann, Mitchell, & Volkman, 2006; Cooke et al., 2001).

A few neuroeconomic studies have investigated the neural correlates of regret when payoffs are monetary (Coricelli et al., 2005; Lohrenz, McCabe, Camerer, & Montague, 2007). These studies find that activity in the medial orbitofrontal cortex (OFC), anterior cingulate cortex (ACC), and right hippocampus is positively correlated with the magnitude of financial regret. Patients with damage to the OFC neither reported regret nor anticipated negative consequences associated with their actions (Camille et al., 2004). How this activation affects decision-making remains unknown, but one possibility is through the propagation of fictive error signals in the ventral striatum (Lohrenz et al., 2007) when the outcome is better than anticipated, or activation in the orbitofrontal cortex when the outcome is worse that expected (Liu et al., 2007).

Thirty-six right-handed volunteers (21 females and 15 males; 18–38 years of age with a mean of 21.64 years) participated in the study. Due to scanner gradient malfunctions, the data from six subjects were only partially acquired, leaving 30 subjects (17 females and 13 males; 18–38 years of age with a mean of 21.3 years) for the data analysis. Each participant was paid USD 40 for her participation. The experiment consisted of 100 trials. At the beginning of each trial, subjects were shown a display containing three doors on a screen located inside the scanner. In each trial, the subject's task was to select one of the doors, which would then be opened. Responses were registered with a fiber optic button-box, placed in the subject's right hand. At the beginning of each trial, the number of doors that contained shocks was indicated at the top of the display. The number ranged from zero to three. The conditions with zero or three shocks served as control conditions with no uncertainty, and we denote these as the no-rejoice and the no-regret conditions respectively. The other two conditions, with shocks behind one or two of the doors, allowed for regret and rejoice to be experienced. After a delay of 6.2–9.8 sec after the decision was made, all of the doors were opened. At the same time that the doors were opened, and depending upon whether a shock was associated with the door selected, the subject was either shocked once or not shocked. After the doors were opened, subjects were required to rate their experience on a horizontal visual analog scale (VAS), as in the experiments described in Sections 2 and 3. The study is described in more detail in Chandresakhar et al. (2008).

We assumed that in trials where the subjects received a shock, they would experience more regret when the probability of the shock was 1/3, followed by 2/3, and by 1 (i.e., monotonic). Similarly, in trials where subjects avoided a shock, they would experience rejoicing. The rejoicing would be strongest when the prior probability of a shock was 2/3, followed by 1/3, and then by 0. We measured fMRI BOLD responses to categorize the regions involved in the experience of regret, of rejoice, and of both experiences. Indeed, the results of the self-reports indicated that participants experienced different levels of regret, as well as different levels of rejoice, in a pattern consistent with our assumptions. Shocks were rated as more unpleasant the less likely they were to be received, and avoidance of a shock was rated more favorably the more likely the shock had been.

Analysis of the fMRI data indicated that individuals experienced a different pattern of brain responses associated with regret and rejoice, and these responses correlated with the degree of the two experiences. A within-subjects ANOVA of the fMRI activation revealed that several regions activated with an intensity modulated by the level of regret (obtained from the interaction term where receiving a shock interacted with the ex-ante probability of it being received). A region was classified as related to regret if it exhibited a pattern of decreasing activity from higher regret to higher rejoice conditions. The set of regret-related brain regions included the left thalamus, left middle occipital cortex, left inferior temporal cortex, right angular gyrus, left precuneus, left superior frontal cortex, and medial orbitofrontal cortex. Activation in the medial orbitofrontal cortex, the left thalamus, the left superior frontal cortex, and the right angular gyrus was greater in high regret than in no regret, as well as greater in no rejoice than in high rejoice. These areas are illustrated in Fig. 4.

Fig. 4.

Activation of Regret-Related Regions, Regret Experiment.

A different network of brain regions was modulated by the level of rejoice. This network consisted of the bilateral ventral striatum, right caudate, left hippocampus, mid-brain/brain-stem, right supramarginal gyrus, right lingual gyrus, left calcarine, bilateral anterior insula, rostral anterior cingulate cortex, and the superior medial frontal cortex. These rejoice-related brain regions showed decreasing activity from higher rejoice to higher regret conditions. A subset of these regions also displayed greater activation with the highest level of rejoice than under no rejoice, as well as greater activation under no regret than the high regret. The subset consisted of the left hippocampus, left ventral striatum, rostral anterior cingulate, and mid-brain. The activation patterns in these regions are displayed in Fig. 5.

Fig. 5.

Activation of Rejoice-Related Regions, Regret Experiment.

Other brain regions exhibited activation levels that increased with both the levels of regret and rejoice. This pattern of activation can be thought of as related to “surprise.” As can be seen in Fig. 6, the right inferior orbitofrontal cortex, pre-supplementary motor area, dorsal anterior cingulate, and the posterior cingulate all displayed this type of pattern. They activated more strongly the lower the prior likelihood of the outcome that was eventually realized, whether the outcome was a shock or no shock. The right amygdala (identified from the main effect of levels) displayed a pattern of activity that did not fit into any of the above classifications. It had greater activation when the possibility of a shock existed. That is, it activated identically under every condition except for the one in which there was zero probability of a shock, in which it exhibited a lower level of activation.

Fig. 6.

Activation of Surprise-Related Regions, Regret Experiment.

Areas showing the strongest relationship included visual processing regions (middle occipital cortex and precuneus), which most likely reflected a shift in visual attention that scaled with the number of alternative outcomes. The thalamus, also associated with visual attention (Kastner & Pinsk, 2004), showed a similar increase in both higher regret and higher rejoice conditions. The other areas associated with regret but not with rejoice included the right angular gyrus, left superior frontal cortex, and the medial orbitofrontal cortex. In a previous study of regret using monetary outcomes, the medial orbitofrontal cortex was implicated in the experience of regret (Coricelli et al., 2005). This study also found that the presence of regret activated the inferior parietal lobule, a region that partially overlapped the area of regret-related activity we observed in the angular gyrus. Liu et al. (2007) found a similar pattern of activation in the orbitofrontal cortex when the outcome was worse than anticipated and striatum when better than anticipated. Thus, it indicates that the activation of some of the regret-related regions, notably the OFC, are robust with respect to a completely different incentive medium – electrical shocks. Moreover, the creation of regret can also occur through variation in ex-ante probability in addition to the variation in magnitude of alternative outcomes.

Avoidance of a shock, leading to the experience of a degree of rejoice, was associated with a distinct network from regret. Surprisingly, the bilateral anterior insula showed the strongest relationship to the degree of rejoice. Although the insula has been linked with painful outcomes (Berns et al., 2006; Brooks & Tracey, 2005; Koyama et al., 2005; Peyron et al., 2002; Ploghaus et al., 1999), its anterior extensions have been characterized as relating to the anticipated emotional state of something potentially painful (Craig, 2003). Its role in rejoice here may reflect the tracking of the ex-ante probability of receiving a shock. More specific to the positive valence of avoiding a shock was the increasing rejoice-related activity of the left hippocampus and bilateral ventral striatum. The striatum, in particular, is generally accepted as playing a key role in reward-prediction errors (Berns, McClure, Pagnoni, & Montague, 2001; McClure, Berns, & Montague, 2003; Montague, King-Casas, & Cohen, 2006; O'Doherty, Dayan, Friston, Critchley, & Dolan, 2003; O'Doherty et al., 2004; Pagnoni, Zink, Montague, & Berns, 2002; Schultz, Dayan, & Montague, 1997; Yacubian et al., 2006). Since we manipulated regret/rejoice through probability, the more likely the anticipated shock, the greater the reward-prediction error when it was avoided. The fact that we observed increasing striatal activity with rejoice only, and not decreasing activity with regret, suggests that the ventral striatum may be involved in only one side of the regret/rejoice continuum. This finding is consistent with a recent study using financial markets which found that this same region computed only positive “fictive error” and not negative ones (Lohrenz et al., 2007). Thus, our finding can be viewed as an extension of this result to non-monetary outcomes.

Regions related to regret and near the striatum, namely the medial orbitofrontal cortex and rostral anterior cingulate cortex, exhibited reverse patterns of activation profiles. The medial orbitofrontal cortex has been implicated previously in studies of regret (Coricelli et al., 2005; Liu et al., 2007), while the rostral anterior cingulate has been previously implicated in emotional and error processing (Bush et al. 2000). Thus the rostral anterior cingulate area which is caudal to medial orbitofrontal cortex may be responsible for the emotional processes related to rejoicing (i.e., pleasure related) and suggests the possibility of a rostral-caudal gradient of different aspects of processing regret similar to that seen in the caudate for reward-prediction errors (Haruno & Kawato, 2006).

“Surprise” is a reflection of the ex-ante probability of a particular outcome: the lower the event's probability, the more surprising when it occurs. This network of surprise-related regions was more widespread than either the pure regret or rejoice networks. The lateral orbitofrontal cortex was the region most strongly activated by surprise. This region has more commonly been associated with aversive outcomes and losses (Hosokawa, Kato, Inoue, & Mikami, 2007; O'Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001; Rolls, 2000; Ursu & Carter, 2005). Its activation with both regret and rejoice in our study may reflect a specialization for the processing of potentially aversive outcomes, whether realized or not. Many of the surprise-related regions we observed, including the posterior cingulate, ACC, and precuneus have been linked to selective attention (Crottaz-Herbette & Menon, 2006; Hahn, Ross, & Stein, 2006; Hopfinger, Buonocore, & Mangun, 2000; Posner & Petersen, 1990; Small et al., 2003), and their activation in our study suggests a potentially augmenting effect of both regret and rejoice. In other words, the emotional response to either form of counterfactual comparison may require more attention to alternative outcomes. Without this attention, it is possible that regret and rejoice are not experienced.