An Argument Against Dual Valuation System Competition: Cognitive Capacities Supporting Future Orientation Mediate Rather Than Compete With Visceral Motivations

Abstract

The dynamic inconsistency of preference is well documented in behavioral research, but its basis remains controversial. In this article, we summarize recent functional magnetic resonance imaging (fMRI) work in the domain of intertemporal choice, specifically considering evidence bearing on the hypothesis that delay discounting in humans is determined by competition between an evolutionarily older system that discounts precipitously with delay (System 1), and a newer system that exhibits very little discounting (System 2). We argue that neuroimaging evidence does not support the hypothesized separate and competing value systems. While it is clear that the sophisticated cognitive capacities that lead to greater valuation of larger later alternatives (e.g., selective attention and self-signaling) depend critically on neocortical structures, these capacities affect intertemporal choice through mediation of (rather than competition with) older cortical and subcortical structures central to reward and motivation. Taken together, neuroimaging evidence supports the alternative hypothesis that intertemporal choice is guided by a single valuation system.

When maximization stopped satisfying behavioral science

Rational Choice Theory (RCT) dominated the behavioral sciences throughout most of the 20^th century (Arrow, 1951). RCT models the individual as maximizing her utility based on a stable set of preferences (Becker, 1976). From the perspective of neuroscience, it is astonishing that this model holds as well as it does; somehow the immeasurable complexity of the living human brain achieves functionality that is not all that far from the rational pursuit of a stable set preferences. That said, systematic irrationality in intertemporal choice and other domains has been well documented and there is growing dissatisfaction with RCT. Of course, demonstration of systematic irrationality is insufficient basis for abandoning RCT; as economist Milton Friedman noted, “Complete realism is clearly unattainable, and the question whether a theory is ‘realistic enough’ can be settled only be seeing whether it yields predictions that are good enough for the purpose in hand or that are better than predictions from alternative theories.“ (Friedman, 1971, page 51)

Attempts to find an alternative that makes better predictions than RCT have generally tried in one way or another to increase the psychological realism of choice models (Edwards, 1953; Kahneman & Tvesky, 1979; Simon, 1967). These efforts incorporate functional constructs intermediate to the contingencies in the environment and the individual’s choice (e.g., emotions and cognitions) that serve as psychological gears within the human decision-making apparatus. The trend has been towards increased collective confidence that the payback in modeling accuracy justifies this massive complication. The rapid emergence of “neuroeconomics” reflects the possibility that we are on cusp of a new development in this program; grounding behavioral theories in neural realism. Perhaps consideration of the physical gears within the human decision-making apparatus will lead to better behavioral science, at least in those areas in which RCT is most faulty. In this article, we look specifically at this issue with respect to intertemporal choice, giving special consideration to the widely discussed idea of dual value systems (Loewenstein, 1996; McClure, Ericson, Laibson, Loewenstein, & Cohen, 2007; McClure, Laibson, Loewenstein, & Cohen, 2004). Whether or not the dual-valuation systems hypothesis is correct, we think it is illustrative of how the neural systems perspective could, in principle, contribute something important to behavioral science.

Intertemporal Choice and RCT

There has been intense investigation of the relationship between immediacy of consequences and motivation (the term “intertemporal choice” is primarily used in neuroscience, “delay discounting” is primarily used in behavioral psychology, and “time preference” is primarily used in economics.) Research on “intertemporal choice” generally requires participants to express preference between a narrow set of alternatives (generally 2 alternatives) that form a conspicuous “sooner smaller” vs. “later larger” contrast (e.g., $5 today or $10 in a month.) Rationality with respect to delay has been conceived in two ways; the first, which we think is in line with common sense, conceives the perfectly rational agent as unaffected by delay per se (though of course opportunity costs and any associated uncertainty are rational considerations by any reasonable standard.) As Jevons put it while characterizing the rational ideal, “all future pleasures or pains should act upon us with the same force as if they were present.”(Jevons, 1871, page 76). The second conception allows that the rational actor devalue delayed outcomes (a preference for immediacy of good outcomes and delay of bad outcomes), provided that the devaluation occurs at a fixed rate per unit of time (Friedman, 1963). Although such “exponential discounting” can be of any rate and is thus, we think, inconsistent with the common sense idea of a rational orientation to delay, it is attractive to Rational Choice theorists because it allows delay to affect motivation, while arguably introducing no violation of RCT’s axioms. Present value for a stream of consumptions (c1, c2, …) discounted exponentially can be represented as :

$$V=\sum _{t=0}^{\infty }{\delta }^{t}u\left(c_t\right)$$ Eq 1

where u is the utility function, with lower values of the discount parameter σ indicating steeper exponential discounting.

Exponential discounting introduces no dynamic inconsistency in preferences. The exponential discounter can make a plan with the expectation that she will follow it unless the circumstances upon which she based her plan shift in an unanticipated way. Her past and future selves harmoniously pursue preferences in an unbroken chain; their success limited only by imperfect information or power. But human and non-humans decision-making does not conform to this standard. On the contrary, there are conditions under which most subjects reverse their initial preference for larger later rewards when smaller sooner alternatives are near at hand. Furthermore, when given the opportunity, subjects choose to lock in their larger later preference while it is still the most appealing (dominant) option in order to protect against an anticipated future self with alternate preferences (Ainslie, 1974; T. Schelling, 1978; Shefrin & Thaler, 1992; Strotz, 1956). In this way, behavioral discounting data look systematically non-exponential, although opinions differ as to the precise alternative functional form that best fits behavioral data (Green & Myerson, 2004). One widely considered alternatives is hyperbolic discounting (discounting that is proportional to delay) (Strotz 1956; Ainslie, 1975, 1992, 2001). Present value for a stream of consumptions (c1, c2, . .) discounted hyperbolically can be represented as :

$$V=\sum _{t=0}^{\infty }\frac{u\left(c_t\right)}{K^{\ast }t+1}$$ Eq 2

where u is the utility function, with higher values of the discount parameter k indicating steeper hyperbolic discounting. A second proposal to account for preference reversals is quasi-hyperbolic discounting. In quasi-hyperbolic discounting, a ballistic devaluation occurs in the presence of any delay, but the effect of additional delay is modeled exponentially (Laibson, 1997). Present value for a stream of consumptions (c1, c2, . . ) discounted quasi-hyperbolically can be represented as

$$V=u\left(c_0\right)+\beta \sum _{t=1}^{\infty }{\delta }^{t}u\left(c_t\right)$$ Eq 3

where u is the utility function, and parameters β and δ both serving as discount parameters. Note that β does not affect the immediate period in the stream of consumption; lower values of β indicate a greater devaluation of all outcomes that are not immediate. The additional effect of delay beyond this discontinuity (δ) is exponential.

In both, hyperbolic and quasi-hyperbolic discounting, the difference in value between immediacy and some delay, d is proportionally greater than the difference between some positive delay, X, and X + d. As a result, both introduce time inconsistency, though for quasi-hyperbolic discounting, this inconsistency is restricted to situations involving immediacy relative to situations without immediacy. As we will consider below, the discontinuity inherent in quasi-hyperbolic discounting lends itself well to a dual-process account of the neural basis of temporal discounting.

The lack of resolution regarding modeling alternatives is related to the enormous variability in discounting observed within individuals across contexts (Chapman, 2000), between individuals in the same context (Ainslie & Monterosso, 2003a), and across different research studies (Frederick, 2002). As a result, there is now considerable skepticism that any model of delay discounting will be consistently adequate. As Roelofsma and Read (2000) put it, “The study of intertemporal choice is currently undergoing a change in emphasis, as has already occurred in the study of decision making under risk and uncertainty. Rather than searching for the holy grail of a single utility function, researchers now take the more pragmatic view that preferences are constructed based on the circumstances of their expression” (Roelofsma & Read, 2000). We will return later to the issue of endemic variability in intertemporal choice behavior.

What does intertemporal choice have to do with impulsivity and self-control? Synchronic multiple selves models

We think that much of the interest in delay discounting relates to the underlying expectation that it bears importantly on self-control problems including overspending, overeating, under-exercising, and addiction. But given the emotionally ‘cool’ nature of the typical intertemporal choice assessment (would you prefer $5 today or $10 in a month?) it is not self-evident that they would be associated with real self-control phenomena. The hypothesized association between intertemporal choice measures and self-control turns on the researcher’s conceptualization of the nature of self-control struggle. With rare exception (Becker & Murphy, 1988), behavioral scientists that model self-control problems either 1) drop the RCT modeling assumption that the relevant agent is a whole unified person and instead model her as the product of conflicting sub-person agents(e.g., Shefrin & Thaler, 1992) or 2) drop the modeling assumption that the agent is unified over time (Ainslie, 1992, 2001; Ross, 2005; Schelling, 1980, 1984; T. C. Schelling, 1978; Strotz, 1956) and instead model her as the product of a temporal series of agents, sometimes engaged in strategic opposition to each other. Following Ross (Ross, In press) we refer in turn to these two classes of self-control models as “synchronic multiple selves” and “diachronic multiple selves” models. We presently focus on synchronic multiple-self models, and will return later diachronic models.

Purely “synchronic” models conceptualize self-control struggle as occurring between distinct ‘sub agents’ within the person, that differentially value the same outcome. Although there is no reason why synchronic multi-self models are restricted to two sub-person agents, most in fact do include just two, sometimes referring to a “System 1” that is evolutionarily older, that is instantiated largely in limbic and paralimbic brain structures, that works faster and without requiring attention, and that is more stimulus-bound and present-focused, and a “System 2” that is evolutionarily recent, that is instantiated in the neocortex, that is generally slow and demanding of attention, and that utilizes more abstract representations of the environment and so is capable of future-oriented behavior (Evans, 2008).

Freud’s ego and id were so conceived, with each operating by its own principles and often at odds. Thaler and Shefrin proposed to capture self-control phenomena with a model of the individual as including “two sets of coexisting and mutually inconsistent preferences: one concerned with the long run, and the other with the short run.” (Shefrin & Thaler, 1992)¹ Loewenstein’s “visceral motivations” model is also in this spirit, since visceral motivations sit apart from non-visceral motivations, and so imply synchronic multiple selves (Loewenstein, 1996). Metcalf and Mischel’s analysis of the “dynamics of willpower” posits multiple agents; one the manifestation of a cool, cognitive “know” system, the other a hot emotional “go” system (Metcalfe & Mischel 1999). The strength-model of self-control (Baumesiter, Vohs, & Tice, 2007) implies a dual system as well, since there is a presumed set of motivations that the “willpower muscle” preferences, and an alternative set that it does not.

From the perspective of these synchronic multi-agent models, delay discounting is generally viewed as one relevant factor, but neither as the primary source of impulsivity, nor as the key to understanding mechanisms of self-control. Instead, in purely synchronic models, time inconsistency is one manifestation of the underlying competition between sub-person agents with competing preferences, and self-control maps directly as the outcome of that competition. In a purely synchronic model of self-control an act entails that a certain sub-agent or sub-agents prevails over others (Freud’s ‘ego’ prevailing over ‘id’, Thaler’s ‘planner’ over ‘doer’, Loewenstein’s ‘non-visceral’ over ‘visceral’ motivations, Metcalf and Mischel’s ‘know-system’ over ‘go-system’, the strength model’s ‘ideals’ over ‘passions.’)

From the standpoint of neuroeconomics, the possibility that there are two separate valuation systems within the brain is attractive. In addition to creating clear intervention targets (e.g., intervening to shift the balance between the systems to treat self-control problems), it suggests that models of individual behavior could be dramatically improved by finding a way to develop two sets of utility functions for the individual. This would not, we think, be mere “neuro-fication” of the timeless rational- Vs emotional-self model, since it would suggest incorporating neuroscience methodologies to efficiently solve for the set of parameters in whatever underlying dual-system utility function was used. For instance, if the dual-system discounting function included a weighting parameter to capture the strength of the contribution of each system, the weighting parameter might be informed by gross neural markers like metabolism, connectivity, gray-matter density, tonic and phasic striatal dopamine, receptor availability, etc.

Neuroimaging research on intertemporal choice: The case for dual valuation

There have been a series of reports combining an intertemporal decision-making task with functional Magnetic Resonance Imaging. Thus far, the highest impact papers among them (Kable & Glimcher, 2007; McClure, et al., 2007; McClure, et al., 2004) have focused on assessing the validity of a synchronic multiple self perspective, and we will discuss these in some detail. McClure et.al 2004 (McClure, et al., 2004) conducted the first study pairing the typical methodology of intertemporal choice investigation with functional Magnetic Resonance Imaging (fMRI). In their study (which had already been cited 450 times at the time of this writing, according to Google Scholar) the rewards at stake for participants were gift certificates for Amazon.com, ranging from $5 to $40. The smaller sooner option varied in delay between the same day (“today”) and 4-week delay; the larger later option was always either 2 or 4 weeks after the smaller sooner option. Subjects received one of their certificates at the specified delay chosen, but did not find out which one until after the end of the test. The analytic approach taken was directed at assessing whether the modeling duality of quasi-hyperbolic discounting (eq 3) might correspond to an underlying duality in neural substrates of valuation. Recall that Quasi-hyperbolic discounting models behavior with two fit parameters, β which represents a ballistic devaluation with any delay, and δ (delta), which represents additional exponential discounting that is continuous over delay. The author’s were interested in the possibility that these two parameters relate to two separate valuation systems; “Our key hypothesis is that the pattern of behavior that these two parameters summarize-[beta], which reflects the special weight placed on outcomes that are immediate, and [delta], which reflects a more consistent weighting of time periods- stems from the joint influence of distinct neural processes, with [beta] mediated by limbic structures and [delta] by the lateral prefrontal cortex and associated structures supporting higher cognitive functions.” (McClure, et al., 2004, pg 504). Rather than seeking to identify regions that reflected value at the time of choice, the analytic approach taken sought to identify regions that were preferentially recruited during choices that included an immediate alternative. Consistent with their hypothesis, this pattern was observed in a set of regions that included parts of limbic and paralimbic systems (“beta regions”). In contrast, lateral prefrontal cortex and posterior parietal cortex were active while subjects were making choices irrespective of delay (and so labeled “delta–regions”). They also observed greater activity in limbic/paralimbic regions in associated with choosing smaller sooner rewards, whereas greater activity in fronto-parietal regions was associated with choices of larger later rewards. Thus the authors viewed their results as suggesting not only the neural basis of delay discounting’s functional form, but also a more general synchronic multiple-selves account of self-control, arguing that behavior is “… governed by a competition between lower level, automatic processes that may reflect evolutionary adaptations to particular environments, and the more recently evolved, uniquely human capacity for abstract, domain-general reasoning and future planning.”(McClure, et al., 2004, page 506)

There is something on its face odd, however, about the idea of a valuation system only concerned with immediate reward that is recruited by the prospect of an immediate gift-certificate to Amazon.com. The participant’s stream of consumption was not likely to change any time soon by obtaining an immediate gift certificate. And the idea that the non-exponential component of discounting is a function of a categorical devaluation of anything that is not immediate is inconsistent with behavioral data (Green & Myerson, 2004) and conceptually problematic since all goal-directed behavior, by definition, entails delayed reward. The fact that the shortest delay used in the study was 2 weeks makes it difficult to discern a continuous effect of delay if discounting is relatively steep (Ainslie & Monterosso, 2004).

In order to address these concerns, McClure et al. conducted a follow-up study (McClure, et al., 2007) that used primary rewards with clear points of consumption; thirsty subjects were asked to decide between receiving smaller sooner and larger later amounts of fruit juice (or water), with delays ranging from 0 to 25 minutes. In addition to quasi-hyperbolic discounting, the researchers considered a “continuous-time generalization” variant of quasi-hyperbolic discounting in which value in the beta system was exponentially related to delay, but at a steeper rate then the delta-system. Present value for a stream of consumptions (c1, c2, . . ) discounted accordingly can be represented as the double-exponential discount function:

$$V=w\sum _{t=0}^{\infty }{\beta }^{t}u\left(c_t\right)+(1-w)\sum _{t=0}^{\infty }{\delta }^{t}u\left(c_t\right)$$

Where u is the utility function, and discount parameter β and δ are bounded between 0 and 1, with lower values indicating steeper exponential discounting for each of the hypothesized systems, and a weighting parameter w, also bounded between 0 and 1, that serves to parameterizing the relative contribution of beta and delta systems. Although discounting is exponential within system, the aggregated result is non-exponential, since the differential rates of the two systems means the net effect of a unit of delay is not uniform across delays. Again they reported fronto-parietal activation that was present during decision-making regardless of immediacy, and limbic and paralimbic activation when an immediate or near immediate reward was possible. However there was little voxel-level overlap in the identified beta-regions across the studies which may be related to different reward types used, or alternatively could relate to timing differences across the studies (Lamy, 2007). Their analysis indicated that the identified beta system did not respond to juice that was delayed by 5 or more minutes, which they point out is odd given that the gift card’s that recruited beta-system activity in their 2004 study were not even physically received until an hour after the experiment (and of course, impacted consumption much later). They conjectured that 1) primary rewards are generally discounted more steeply than secondary rewards (an idea with considerable empirical support as reviewed in Ainslie and Monterosso 2003 (Ainslie & Monterosso, 2003b) )and 2) secondary rewards are likely to be more subject to framing effects than primary rewards, such that framing the Amazon gift card as immediate recruits the beta system even though the realization of the reward is not literally immediate. Although we think this is plausible, we believe that the idea that framing effects are reflected in beta system activity hints at an integration between delta and beta systems that contrasts with the dual valuation model as articulated by the authors. We will return to this idea subsequently. Unlike in the previous study, the investigators did not find evidence that activity in beta- and delta-systems predicted behavior, when controlling for the content of the alternatives, but nevertheless conclude that their findings support the idea that intertemporal choice, and behavior more generally, “reflects the interaction of qualitatively different systems” (McClure et al, 2007, pg 5802).

Neuroimaging research on intertemporal choice: The case against dual valuation

A critical objection raised with regard to McClure et al.’s interpretation of their data relates to the issue of value (Kable & Glimcher, 2007). In McClure 2004, the authors interpret greater activity recruited by choices involving an immediate alternative as evidence that the region is part of the beta system. However, the alternative single value-system account also predicts that choices that include an immediate alternative will recruit more activity than those that do not in any region associated with value. Since the absence of delay results in higher valuation for the same monetary amount, without taking additional steps to offset the discounting effect, pairs with an immediate alternative will tend to have higher value than pairs without. Thus theorists favoring a single value system would suspect that the observed “beta regions” are actually all regions associated with value, and observed delta regions are those related to processes apart from valuation that are involved in performing the task (Kable & Glimcher, 2007). Presumably it was with this objection in mind that a secondary analysis was included in McClure et al. 2004, in which the primary contrast was repeated after controlling for estimated value, with the main results unchanged (see footnote 29). However, value in this secondary analysis was estimated using an exponential discount rate (7.5% per week). If the actual discounting with delay was hyperbolic, their procedure would tend to leave positive residual value in choice pairs that included an immediate alternative. And so their re-analysis did not rule out the most plausible single-valuation system model.

Kable and Glimcher 2007 reported on another intertemporal choice fMRI study, in which the results did not suggest any duality of valuation (Kable & Glimcher, 2007). In this study, the immediate reward was $20 now in all trials, and delayed options were constructed using one of six delays (6h-180d) and one of six amounts ($20.25-$110). Delays were the same for every subject but changed across sessions; amounts were individually chosen for each subject based on the previous behavioral sessions to ensure an approximately equal number of immediate and delayed options were chosen. The authors took a “psychometric-neurometric” comparison approach in which they assessed whether delay and amount (external variables) influence both psychophysical and neurobiological measurements in a similar manner (Kable & Glimcher, 2007). Participant’s behavioral data were well-captured by hyperbolic discounting model (eq 2). Moreover, when neuroimaging data were compared to value inferred from behavior and modeled using eq. 2, there was a striking correspondence. Increases in activity in limbic and paralimbic regions including the ventral striatum, medial prefrontal cortex and posterior cingulate cortex (the beta-system regions of McClure et al. 2004, 2007) appeared to track subjective value. The authors conclude that their findings, falsify the hypothesis that the limbic and paralimbic regions form an impulsive neural system that contributes one of two inputs relevant in overall valuation; “[beta regions] do not even primarily value immediate rewards, as the value implied by neural activity is not more impulsive than the person’s behavior, as the beta-delta hypothesis requires.” (Kable & Glimcher, 2007, page 1631). Rather these data appear to show that the “beta system” identified in McClure et al. (2004) track subjective value at all delays. Further, although not directly measured in these data, we believe these data suggest greater integration between the limbic system and the higher-order associative cortical structures that support good decision-making in human (Damasio, 1994). This hypothesis can be more directly examined using connectivity analysis (see Hare, Camerer and Rangel, 2009.)

Framing effects and self-signaling: The basics of a diachronic multiple self model

The “value-integration” alternative to dual-valuation is compatible with the possibility that there are important dissociations related to neural systems that will inform behavioral science. However, value-integration entails that those dissociations feed ultimately into a single motivational system that ultimately selects action (Montague & Berns, 2002). Here we briefly develop a diachronic multiple-selves model, and suggest some implications for neuroeconomics.

In contrast to synchronic multiple self models, purely diachronic multiple self models do away only with the idea that the agent is consistent over time. As a result of non-exponential discounting the individual’s preferences at any moment in time may be threatened by her future self’s foreseeably conflicting preferences (Ainslie, 1992, 2001; Ross, 2005; Schelling, 1980, 1984; T. C. Schelling, 1978; Strotz, 1956). She can at one moment prefer to lose weight, while at the same time know that her future self may choose to overeat. She may take “precommiting” action to secure her current preference, for instance spending her money on a wardrobe two sizes too small, or undergoing gastrointestinal bypass surgery. But such actions are last resorts – the approach that is likely tried many times before precommitment is to simply resolve to a particular course, such as eating less. “Resolving” is not merely planning, in that it entails something more – something directed at the foreseen uncertainty that one’s own future self will abide. We think it is not possible to develop a reasonable program of research for neuroeconomics in the area of intertemporal choice and of self-control without simultaneously developing a functional account of “resolving”. We believe there is a compelling account (Ainslie 1975, 1992, 2001; Benabou, Roland, & Tirole, 2004; Bodner, Ronit, & Prelec, 2003) to which we now turn.

Recall that the findings of McClure 2004 suggest that framing makes a difference in the effect that delay has on motivation; Amazon gift cards available “immediately” recruited substantially more limbic activity than those available a few days subsequently. What if a similar experiment were conducted that framed immediate gift-certificates as ‘prizes that could be received in the mail in a few days’ and what if that experiment also included other prizes that could be selected immediately from a lab store? Based on previously reported effects in the behavioral discounting literature (Lowenstein, 1988) and of fMRI research looking at framing effects related to Prospect Theory (Breiter, Aharon, Kahneman, Dale, & Shizgal, 2001), we expect that the “immediate gift-certificate” would illicit less activity in regions associated with reward value, which is to say we agree with the interpretation offered in McClure et al. 2007 that their earlier findings related to the framing of some rewards as immediate. Framing effects that violate RCT are easy to generate (Kahneman & Tvesky, 1979). Importantly, framing effects need not be imposed upon the individual. As elegantly demonstrated in work on “mental accounts” (Thaler, 1980), the individual actively generates her own framing effects. For example an individual may decide that she will treat one source of money as untouchable, and another source as “fun money”, despite the reality that money is fungible.

Self-generated framing effects take us one step in the direction of a diachronic multiple-selves model of self-control. Given the realization that one’s future self may thwart current preference, self-generated framing effects can be a strategic gambit. For example, someone struggling to save money may cultivate a “penny saved is a penny earned” orientation by keeping track of money saved relative to her former careless spending habits. Strategic framing effects can also capitalize on the fact that goal-directed behavior can be conceived on a continuum from the molecular (e.g., trying to hit a nail with a hammer) to the molar (e.g., trying to build a house) (Howard Rachlin, 1995) . Indeed self-control conflict can be conceived as a clash between preference at molecular and molar levels (Aristotle, 1984). For example, one may want this particular cigarette (a molecular framing), but prefer being a non-smoker to a smoker (a molar framing). While it has been argued that the distinction is essentially cognitive, with molar level conception something that is learned (H. Rachlin, 2000), a more plausible alternative is that in a time where temptation is low, the individual may make an effort to “stamp-in” a global frame for her future self. For example, a mother trying to quit smoking, in a moment when the desire to quit is strong, may decide to frame abstinence as an expression of her love for her child. In so doing, she is attempting to cast the contingencies her future self experiences in a highly global frame. Resolutions can be viewed this way; a resolution to quit smoking places any subsequent particular cigarette (which alone carries minimal health cost) into a context in which, if the frame sticks, the goal of being a non-smoker is implicated by the particular decision of whether to smoke the next tempting cigarette (Monterosso & Ainslie, 1999)

But what is the force that serves to make a strategic self-generated framing effect stick? What prevents the fore-mentioned mother from deciding to reframe the craved cigarette at a molecular level (just this one)? Often this is, we think, precisely what happens. But sometimes resolutions succeed. Momentum may play a role (Nevin & Grace, 2000; H. Rachlin, 2000), but the potential reward for abandoning a molar frame may be very high, and so there needs to be more force to explain how resolutions ever succeed. The key idea, as formulated by Ainslie (Ainslie, 1975, 1992, 2001) is that the behavior cannot help but be treated by the individual as a test-case. If I have, as Mark Twain quipped, quit smoking hundreds of times, and I recently resolved to never smoke again, is it credible that I will quit tomorrow if I break yesterday’s resolution and smoke today? Having strategically framed the contingency in the past, the individual may be saddled with the conception that current behavior holds importance to the molar goal. Economists have incorporated this idea into formal modeling under the label of “self-signaling” (Benabou, et al., 2004; Bodner, et al., 2003). The idea is that, in part as a function of one’s history of resolution-making, the individual case takes on additional meaning and utility. In addition to the benefits and the cost of one cigarette, my own behavior serves as a signal that informs my expectation of whether I will be a smoker in the long-run. The probabilistic signal, multiplied with the utility of whatever the signal refers to, thereby contributes to the overall utility. So if one’s framing of not smoking as an act of motherly love prevails, then smoking takes on considerable additional disutility.

A complication here is, we think, that the cost of the frame is not independent of its likelihood of prevailing. When there is a high reward to be gained by molecular framing of the contingencies, there is a greater chance that molecular framing will prevail. And so the model includes an internal feedback between framing and valuation that is difficult to model. While unfortunate for behavioral scientists, this cannot be taken as evidence against its correctness.

From this point of view, there are at least two important places to consider in developing the neuroeconomics of self-control. There is 1) behavior at the time of resolving, and there is 2) behavior at the time at which prior strategic behavior may or may not have its effect (that is, the time of temptation). We think both of these are dependent upon the sort of sophisticated cognitive capacities that get classified as “executive functions” and that depend heavily upon multimodal associative cortex. Any resolution depends on sophisticated cognitions that if not linguistic, are at least related to the abstraction that makes language possible. A resolution requires abstracting a category (e.g., smoking), holding a preference with regard to the category, and foreseeing its vulnerability. And of course, all strategic reframing deployed at the time of making a resolution is not equal – some may even backfire (Ainslie, 1992). At the time of temptation, we think again executive functions are likely to play a critical role. Molar and molecular frames are likely asymmetrical with respect to their dependence upon high-order cognition, because molecular frames (e.g., this one cigarette) are more directly connected to the stimulus (e.g., the cigarette in view), and so are boosted by bottom-up associations. Molar frames (e.g., interpreting smoking as related to one’s commitment to motherhood) get less help from the immediate stimulus and so depend upon remembering, and perceiving the applicability of past resolutions. In addition, there may be an important role for interference control, whereby top-down processes bias attention to prevent a currently dominant molar framing of the contingencies from being overturned by the local framing that would otherwise be suggested by bottom-up associations. It was this that William James had in mind when he wrote (perhaps overstating the relevance of mere attention,) “Effort of attention is thus the essential phenomenon of will. Every reader must know by his own experience that this is so, for every reader must have felt some fiery passion’s grasp. What constitutes the difficulty for a man laboring under an unwise passion of acting as if the passion were wise? Certainly there is no physical difficulty.… The difficulty is mental: it is that of getting the idea of the wise action to stay before our mind at all.” (William, 1890).

The above account of self-control makes many predictions that are in line with some of those made by the beta-delta account. On the above, self-control is expected to relate in part to one’s capacity for high-level cognition (a proposition that is supported both by the developmental literature and by the individual difference literature). Moreover, taxing of high-order cognitive capacity with a dual task manipulation or through fatigue (Baumeister & Heatherton, 1996), or through chronic drug use (Goldstein & Volkow, 2002) would be expected to lead to self-control lapses. But while the predictions are highly overlapping with those of the beta-delta/ System 1 Vs. System 2 models, the present model maintains that higher-level processes have their impact by brokering reward within a single motivation marketplace. We think this account is consistent with the exciting recent finding reported by Hare, Camerer and Rangel 2009 (Hare, Camerer, & Rangel, 2009) that health considerations only sometimes affected choice and signal change in the VMPFC, and whether or not they did was at least statistically mediated by the presence of more activity in the DLPFC.

Distinguishing specified contingencies from prevailing contingencies

The perspective on self-control outlined above suggests a methodological problem for behavioral scientists using the delay-discounting construct. As discussed above, the enormous variability within and between subjects has suggested to some that there is no single function that can represent a subject’s basic discount rate (Loewenstein & Prelec, 1992). Human delay discounting varies markedly from one domain to another in the same individual, so that, for example, an individual’s level of discounting money scarcely predicts her level of discounting of health outcomes (Chapman, 1996). This undercuts the idea that estimates of delay discounting derived from a monetary choice procedure (or any other single reward procedure) provide a general index of impulsivity that can be used to predict behavior in other domains (Roelofsma & Read, 2000). The framing effects discussed above may be the underlying basis of the problem. Among humans the contingencies perceived to be at stake go well beyond the outcome of the choice at hand. Just as resolutions and other strategic framing effects may lead to the formation of rules about smoking, they can lead to tacit or explicit rules about monetary trade-offs between delay and amount, choices between short term and long term personal relationships, and even judgments about whether an impulse is consistent with the person’s perception of her character. When a person structures her choices in relation to resolutions or more subtle strategic reframing, she can be expected to express different preferences than she would if she were making a choice just on the basis of its own merits, and these preferences are apt to differ among categories of reward, according to their temporal distribution, emotional relevance, impulse control history, and doubtless many other factors. The experimenter is blind to all this, and the computation of k (or whatever parameters are included in the investigator’s model) assumes unrealistically that the prevailing contingencies are precisely those specified by the alternatives presented to the participant. A subject’s choice of $10 delayed by a month over $8 today may carry the additional utility of signaling to the self that she is a financially responsible person, but there is no easy way to incorporate this framing effect, and so the researcher is left inferring that her discounting must occur in a way that leaves $10 retaining more than 80% of its value given 1 month delay. The reality may be that in the absence of framing effects, the immediate alternative would be more valued. Related, in our lab we are currently looking at the possibility that valuation assessed in a non-choice context (where presumably, such self-signaling is not applicable) shows greater delay discounting than appears when inferred based on choices. That is, we are interested in the possibility that, for instance, signal change in brain regions that respond in step with value might indicate greater value for the immediate $8 than the delayed $10 in the same participant that choses the delayed $10.

The future of interteporal choice neuroimaging research: Enough with the money already

As summarized in Table 1, we know of 11 papers relating intertemporal choice behavior to brain activity as indexed by function Magnetic Resonance Imaging. In brief, some other findings of note include associations between variability of activity on individual trials and choice, with 1) greater neural activity in PPC, DPFC and rPHG associated with choosing more immediate rewards (Boettiger, et al., 2007) while greater activity in LOFC (Boettiger, et al., 2007) and insula (Wittmann, Leland, & Paulus, 2007) were associated with choosing delayed rewards. Between subject variability in task-related activity was also reported to be associated with between-subject variability in discounting; those that discounted more steeply exhibited more recruitment in the ventral striatum in response to gains and losses (Hariri, et al., 2006). In another study, individual degree of delay discounting was associated with and with recruitment in rostral anterior cingulate cortex when rating the current vs future self (Ersner-Hershfield, Wimmer, & Knutson, 2008). An inverse correspondence between individual differences in delay discounting and signal change during intertemporal choice was reported in inferior frontal cortex (Wittmann, et al., 2007; Monterosso et al., 2007), left anterior prefrontal cortex(Shamosh, et al., 2008), and neucleus accumbens, dorsoleral prefrontal cortex, posterior cingulated cortex and medial prefrontal cortex (Ballard & Knutson, 2009).

Table 1.

Results from fMRI studies directly related to intertemporal choice. The following codes are used. Pop: NM: Normal; ALC: sober alcoholics; ADHD: attention deficit hyperactivity disorder; MA: acutely abstinent methamphetamine dependent Reward: $: money for one or more of the trials; Hyp $: purely hypothetical money; JUICE: juice and water for thirsty subjects Scanner task: ITC: inter temporal choice task; 2-CH: 2-choice prediction game; WM: Working Memory Task

	Method				Results
Paper	N	Pop	Reward	Scan Task	Associations with individual choices (within subjects)	Associations with degree of discounting (between subjects)
1 McClure et.al 2004	14	NM	$	ITC	Regions associated with choice of LL: Lateral PFC and parietal activity Regions associated with choice of SS:limbic, paralimbic regions > Greater limbic and paralimbic activity in presence of immediate option *See text above re caveat	NA
2 McClure et.al 2007	37	NM	JUICE	ITC	Greater limbic and paralimbic activity in presence of immediate option	NA
3 Kable and Glimcher 2007	12	NM	$	ITC	A match between psychometric and neurometric discount functions in regions tracking value (ventral striatum, medial PFC & posterior cingulate cortex)	NA
4 Boettiger et.al 2007	19	ALC	$	ITC	NA	Regions related with steeper delay discounting: R supramarginal gyrus, R cerebellum, R middle temporal gyrus, R inferior temporal gyrus, R parahippocampal gyrus, L superior frontal gyrus; Regions associated with shallower discounting: R inferior frontal gyrus
5 Wittmann et.al 2007	13	NM	HYP $	ITC	Regions associated with choice of LL: LR posterior insular cortex, L posterior cingulate, L superior temporal gyrus, L angular gyrus, L inferior parietal lobule,L cuneus	Regions associated with steeper discounting: R posterior cingulated, L lingual gyrus, R cuneus, R caudate, L superior temporal gyrus, R lingual gyrus, L inferior frontal gyrus (BA 45). Regions associated with shallower discounting: LR inferior frontal gyrus (BA 47)
6 Plichta et.al 2009	26	ADHD	$	ITC	NA	Hypoactivation in VS among ADHD Hyperactivation of amygdala recruited with delay in ADHD
7 Hariri 2006	45	NM	Hyp $	2-CH	NA	Individual differences in delay discounting correlate positively with magnitude of ventral striatum in response to reward.
8 Ersner-Hershfield et.al 2008	18	NM	$	FT	NA	A positive correlation between delay discounting and individual differences in current vs future self activation at rACC.
9 Monterosso et al 2007	29	MA	$	ITC	None Observed	Regions associated with shallower discounting: L ventrolateral prefrontal cortex
10 Shamosh et.al 2008	103	NM	Hyp $	WM	NA	Regions associated with shallower discounting: Working memoryrelated neural activity in left anterior prefrontal cortex
11 Ballard and Knutson 2009	16	NM	$	ITC	NA	Regions associated with shallower discounting: Nacc (tracking magnitude) and DLPFC, PCC and MPFC (tracking delay).

Other than the McClure 2007, all the above used money as reward. And without exception, all were based on decision-making between a set of 2 alternatives that formed a conspicuous “sooner-smaller” vs. “later-larger” set (e.g., $5 today or $10 in a month). If, as we think sensible, one considers all situations in which contingencies involve varying delays to be “intertemporal choices,” then the topic is far broader than this. Consider for example, the idea of “social capitol” -- the potentially rich benefits associated with the network of social relationships that humans maintain (Bourdieu & Richardson, 1986). Every interaction, or avoidance of interaction, affects the individual’s social capital. Does she inhibit annoyance when feeling irritated? Does she indulge in the immediate gratifications of bragging? Does she lie for immediate protection at the risk of a long lasting cost to reputation? In general, although the trade-offs are vague and uncertain (what exactly is the cost of being thought a braggart, and when is that cost paid?), the individual’s orientation to delay is de facto incorporated into her behavior. At the extreme, an individual with no concern for the future will have no reason to exercise the restraint required to maintain social relationships. The conspicuous SS vs LL choice is, we think a relatively rare case. And while monetary rewards may make life easier on researchers and may bring particular elements of function to light, it is not likely that experiments based on monetary trade-offs are inclusive of all the factors relevant in intertemporal choice more broadly conceived, and they may well be idiosyncratic. Indeed, $8 now Vs. $10 in one month may be treated as a math problem for some subjects; intertemporal choices that do not involve well specified contingencies within a single domain (such as whether or not to refuse an inconvenient request from a friend) could not be so treated. We suspect that this accounts for the disconnect between the neuroeconomic literature on delay discounting and the clinical lesion literature which implicates the VMPFC as a critical substrate of far-sighted behavior (Bechara, Damasio, & Damasio, 1994; Bechara, Damasio, Tranel, & Damasio, 1997; Damasio, 1994). Again, we think the recent work by Hare and colleagues (Hare, et al., 2009) in which appetite and interest in health were used as the contingency domains provides a compelling example of where investigation of intertemporal decision-making will move in the near future. With respect to the dual-valuation theory, we think the current evidence suggests that although self-control relies on sophisticated cognitive processes supported by evolutionarily recent associative cortex, that its effect is realized within a single underlying market place of valuation(Ainslie & Monterosso, 2004).