The behavioral economics of will in recovery from addiction

Abstract

Behavioral economic studies demonstrate that rewards are discounted proportionally with their delay (hyperbolic discounting). Hyperbolic discounting implies temporary preference for smaller rewards when they are imminent, and this concept has been widely considered by researchers interested in the causes of addictive behavior. Far less consideration has been given to the fact that systematic preference reversal also predicts various self-control phenomena, which may also be analyzed from a behavioral economic perspective.

Here we summarize self-control phenomena predicted by hyperbolic discounting, particularly with application to the field of addiction. Of greatest interest is the phenomenon of choice bundling, an increase in motivation to wait for delayed rewards that can be expected to result from making choices in whole categories. Specifically, when a person’s expectations about her own future behavior are conditional upon her current behavior, the value of these expectations is added to the contingencies for the current behavior, resulting in reduced impulsivity. Hyperbolic discounting provides a bottom-up basis for the intuitive learning of choice bundling, the properties of which match common descriptions of willpower. We suggest that the bundling effect can also be discerned in the advice of 12-step programs.

Recovery from addiction is a distinctly human phenomenon (Logan, 1993), can be extraordinarily abrupt without any obvious changes in contingencies (Premack, 1970, Miller and C’de Baca, 2001), and is commonly described in spiritual terms (Bien and Bien, 2002). Thus it may seem that although behavioral economic and other reductionist approaches are productively applied to the onset of addiction, they are not applicable to studying recovery from addiction (for example, see Miller’s 2003 discussion of his skepticism). Here we want to make the case that behavioral economics sheds new light on recovery from addiction.

1. Limitations of rational choice theory

A great deal of research has been done on addictions and their treatment, but the reason why addictions occur in the first place is still unclear. The one undeniable element common to addictions is the addict’s inability to escape when she wants to, that is, the persistence and frequent alternation of contradictory preferences about her addictive activity. Her inability to make her choices consistent over time is what compels her to seek treatment. Indeed two of the seven defining criteria for substance dependence in the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) explicitly incorporate this dynamic inconsistency: “The substance is often taken in larger amounts or over a longer period than intended.” And “There is a persistent desire or unsuccessful efforts to cut down or control substance use.” This property alone makes addiction anomalous in terms of the utility theory that is the tacit norm of most behavioral sciences, the rational actor model or rational choice theory (RCT; e.g. Boudon, 1996; Gintis, in press; Sugden, 1991). According to RCT, addiction has no motive to occur; a person with enough information and time to assimilate it should arrive at hierarchies of preference that are internally consistent (transitive, commensurable, etc.), that maximize her probability of getting what she prefers, and that do not shift as the perspective of time changes (that is, are dynamically consistent).

Such anomalies are no longer thought to be rare, however; in recent years violations of RCT have been described in abundance. Jolls et.al 1998 summarized them in three categories: bounded willpower (a failure to follow your own plans), bounded rationality (failure to correctly interpret environmental contingencies) and bounded self-interest (a tendency to invest altruism where it will not bring returns). The greatest amount of research has been done on bounded rationality (e.g. Tversky & Kahneman, 1981; Camerer, 2004) most of which is peripheral to the problem of addictions, as is bounded altruism. There has been much less research on bounded willpower, a concept that lacked any scientific standing until a generation ago (Ainslie, 2001, p. 202, footnote 12). Following initial suggestions that systematic deviations from constant rates of discounting the future might lead to reversals of preference as a function of time alone (Strotz, 1956, Ainslie, 1975), several economists have studied this pattern in poor financial self-control (well represented in Camerer et.al., 2004), and several psychologists have done research on it in addicts (see section 2.1 and Bickel & Marsch, 2001). Both economists and psychologists have described external commitment devices that approximate self-control, but only a few (economists O’Donoghue & Rabin, 2004, psychologists Bodner & Prelec, 2001) have explored the theoretical possibility that purely intrapsychic self-control can arise from the same deviation that makes it necessary (first described as private side-betting—Ainslie, 1975). None have tested this possibility empirically, for the simple reason that it entails a recursive process, which cannot be subjected to controlled experiment. However, several kinds of observation have increased the probability that this strategy of self-control exists and is the mechanism of “willpower.”

Our plan is to describe the well-established tendency to deviate from constant discounting, and present evidence that it permits not only addictive choice but also a countervailing tendency toward self-control. We will also suggest how the properties of this self-control can be discerned in the “12-Step” approach that is the modal treatment for addiction in the United States, and increasingly prevalent world-wide. Finally, we will discuss briefly objections that have been raised to this model of dynamic inconsistency.

2. Hyperbolic discounting as a factor in addictive behavior

The immediacy of reward associated with drug use clearly has some importance in understanding why quitting can be difficult. The rewards from drug use are immediate and the adverse consequences tend to be delayed; were the reverse true-- if the high from smoking crack didn’t arrive till weeks later, but the bad feelings associated with all the personal loss came on as soon as blood carried the drug to the brain-- it is unlikely there would be problem drug use. As Samual Butler remarked “If the hangover preceded the intoxication, alcoholism would be a virtue.”

The intuition captured in Butler’s quip suggests that the effect that delay has on the motivational potency (“value” in RCT’s terms) of expectancies may be a source of the apparent violation of dynamic consistency that is fundamental to addiction. While it has been held by some economists that any tendency to devalue expectancies as a function of mere delay is inherently irrational (Pigou, 1920), rational maximization axioms do not proscribe delay-discounting; they do require that devaluation occurs as an exponential function of delay. That is, RCT assumes that delay discounting occurs at a constant rate per unit of delay, in the same way a bank pays interest on a balance per unit of time. The critical feature of exponential discounting from the standpoint of rational maximization is that it preserves dynamic consistency. If delayed outcomes are devalued exponentially, then the relative preference of expected future outcomes does not change as the individual moves closer in time to those outcomes, just as the relative size of bank balances growing at the same interest rate never changes, regardless of when the accounts were opened.

In contrast with RCT’s assumption, findings from behavioral economics provide compelling evidence that delay discounting does not occur by a fixed rate per unit of time, and instead occurs in a way that implies dynamic inconsistency. In typical human experiments, delay discounting is inferred by asking participants to choose between alternatives that vary in both amount and delay, such as, “Would you prefer eight dollars (or euros, etc.) today or eleven dollars in one month?” If an individual chooses $11 in one month over $8 today, but also prefers $8 today over $9 in one month, then she might be estimated to be indifferent between $8 today and $10 in one month. A set of such inferred indifference points over a range of delays allows the estimation of an overall delay discount function relating delay to value. Such studies have been carried out with choices between points on a counter (Forzano and Logue, 1994), hypothetical health outcomes (Chapman, 1996, Chapman, 2000, Chapman et al., 2001, van der Pol and Cairns, 2001), hypothetical drug or alcohol (Madden et al., 1997, Madden et al., 1999, Petry, 2001, Bickel et al., 1999), hypothetical money with context (Thaler, 1981, Chapman, 1996, Chesson and Viscusi, 2000, Bohm, 1994), hypothetical money without context (Fuchs, 1982, Ainslie and Haendel, 1983, Madden et al., 1997), actual money (Crean et al., 2000, Ainslie and Haendel, 1983, Kirby and Herrnstein, 1995, Richards et al., 1999, Wallace, 1979), consumer goods (Kirby and Herrnstein, 1995), food (Kirby and Herrnstein, 1995, Mischel et al., 1969, Mischel and Grusec, 1967, Forzano and Logue, 1994); and juice (Logue et al., 1990). Less frequently, choices among punishments have been used as well, including shocks (Cook and Barnes, 1964, Hare, 1966, Mischel et al., 1969) and aversive noise (Navarick, 1982).

The experimental evidence from the above studies indicate that, like other psychophysical relationships, the relationship between expected delay and valuation is proportional or hyperbolic (Ainslie, 1975). Hyperbolic discounting implies that the increase in valuation that occurs when moving a fixed unit of time closer to an expected outcome is proportionately greater the closer one is to that outcome. Think of the experience of waiting for an additional day for an important event that is a year off, versus for one that is imminent. The spike in value (positive or negative) as one gets temporally closer to a particular reward creates systematic reversals of preference over time. In the morning the prospect of being hung over the following day may be dominant over getting drunk that night. But come nighttime, with the pull of drinking immediate and the hangover still hours off, preference may switch. Unlike exponential discount functions, hyperbolic discount functions predicts dynamic inconsistency (Form 1 and 2; Fig 1a and 1b).

Figure 1A and 1B.

Exponential discount curves from two rewards of different sizes available at different times (1A), and hyperbolic discount curves from two rewards of different sizes available at different times (1B). For exponentially discounted rewards there is no delay at which preference switches. For the hyperbolically discounted rewards, the smaller reward is more valued just in the period when its availability is relatively immediate.

The exponential formula is:

$$\text{Value}=\text{Amount}\times {(1-\text{Rate})}^{\text{Delay}},$$ (Form. 1)

where Rate is a percent amount per unit time.

The hyperbolic formula is:

$$\text{Value}=\text{Amount}/(1+\text{K}\times \text{Delay}),$$ (Form. 2)

where K is a parameter reflecting impatience.

2.1 Empirical studies of discounting in addicted populations

Hyperbolic discounting, unlike exponential discounting, creates windows in which a more immediate but inferior reward is temporarily preferred over its alternative (impulsiveness-- Ainslie, 1975). However since most individuals are not impulsive to the point of substance abuse, some additional factor must be at work. People who become addicted may get greater or longer pleasure (or relief) from their substances, or less pleasure from the alternatives. They may also discount future rewards more sharply and/or have less skill in leveraging that motive into consistent behavior. We will deal here with only the latter two mechanisms

Across a range of addicted populations, the evidence has been at least consistent with the hypothesis that steep discounting is a factor in substance abuse. Relative to control participants, significantly steeper delay discounting has been observed in a heterogeneous group of substance-dependent subjects (Ainslie and Haendel, 1983), heavy social drinkers and problem drinkers (Vuchinich and Simpson, 1998), smokers (Mitchell, 1999, Fuchs, 1982, Bickel et al., 1999), methamphetamine dependent individuals (Monterosso et al., in press), and opiod dependent individuals (Kirby et al., 1999, Wallace, 1979, Madden et al., 1997, Madden et al., 1999, Bretteville-Jensen, 1999). Also of interest, heroin addicts who shared needles have been observed to discount money more steeply than heroin addicts that did not share needles (Odum et al., 2000).

However, the finding of steeper delay discounting among addicted populations does not necessarily imply that this discounting is a causal factor in addiction. Addicted and non-addicted populations are self-selected, and so are liable to differ in myriad ways other than their drug use -- ways that could drive an association between delay discounting and substance abuse without the two being causally linked (Meehl, 1970). Moreover, to whatever extent delay discounting and substance abuse are causally linked, it is indeterminable from cohort comparisons in which direction the causal arrow points. It is possible, for example, that the effects of chronic drug use on the brain (Volkow et al., 1988, Volkow et al., 1991, London et al., 1990, Kosten, 1998) might affect performance on delay discounting tasks. Or, perhaps the life-style of the addict might dispose her to emphasize immediate attainment of reward (a possibility we will consider further in the context of our discussion of willpower below). All said, the existence of consistent steeper discounting among addicted populations is consistent with, but does not prove, the hypothesis that individual differences in temporal discount rate are responsible for addictive behavior.

Whether or not steeper discount rates make self-control harder for some individuals, the hyperbolic shape of the curves predicts that a person’s preferences will be inherently inconsistent over time. Accepting this, the tendency to choose rewards that are inferior from a distance (as characteristic of addictions) is not the phenomenon in need of explanation. If delay discounting is hyperbolic, it is the consistency sometimes achieved (the recovered alcoholic, or simply the wise investor) that requires an explanation. We turn now to phenomena of self-control that emerge from hyperbolic discounting.

3. The self as population

Hyperbolic discounting transforms what has been modeled as a unitary self into a population of competing agents. The elementary agents are no longer individuals, but motivated processes within the individual—call them interests—which compete to be “her” choice on the basis of the shifting values of the rewards on which these interests are based. Allowing for hyperbolic discounting in economic utility functions radically changes behavioral economic modeling in ways that have so far received little attention from the addiction research community. Given some sets of alternatives, hyperbolic discounting predicts that an individual will hold preferences that she can foresee will be threatened by her own future self. An interest based on career success, say, might be unable to compete with the nightly surge of an interest in heavy drinking, and the intermittent victory of the drinking interest might progressively weaken the interest in career success. The individual who in the morning prefers to not get drunk may be aware that this preference is in danger of being thwarted by her self in the evening, and further, that left unchecked, this reversal of preference will lead to mounting regret as the daily hangovers impair performance. For hyperbolic discounters, future selves are potential obstacles against which she has reason to act strategically. The conventional concept of the unitary individual is thereby altered to that of a constrained competitive marketplace, in which incompatible interests compete for dominance. The strategic interactions that are available to the dynamically inconsistent agent have been most directly explored in a branch of behavioral economics we call picoeconomics (Ainslie, 1992, Ainslie, 2001, Ainslie, 2005), but it has been treated by other authors as well (Elster, 1999, Laibsen, 1997). We include a partial summary here, emphasizing how the strategic interaction of successive motivational states is apt to influence the phenomena of addiction.

3.1 Precommitment mechanisms

Strategic competition between mutually exclusive interests (e.g., intoxication versus sobriety) occurs where a smaller-but-sooner good, when imminent, is more valued than its larger but later alternative. The most straightforward strategic action that can be taken by the present self to forestall an anticipated reversal of preference is to find a method to precommit to the currently desired alternative. The methods of precommitment available depend in part on how imminent a reversal in preference is. The simplest occur when a predictable preference reversal is sufficiently far off that conscious deliberation and action can be undertaken. Precommitment might take the form of fully eliminating the alternative that is currently unwanted but apt to be preferred by one’s future self, for instance closing the option for drug use at least in the short run by checking-in to a remote treatment facility. Pharmacologic agents such as naltrexone and buprenorphine reduce the high available from some substances, and the hoary disulfiram (Antabuse) adds the threat of nausea besides. Drug vaccines, which create antibodies that bind to drugs and prevent them from crossing into the brain, may eventually be a means by which an individual can eliminate the possibility of a future self getting high. More often, though, precommitment takes the form of action that can be expected to make the currently unwanted alternative less desirable. This is perhaps most commonly effected by announcing a resolution, thereby creating the disincentive of embarrassment or social stigma if one fails to follow through on the resolution. More subtle methods of partial precommitment are available, even when physical action is impractical. For example, the alcoholic struggling to maintain abstinence may direct attention away from activities that lead to drinking, or cultivate an emotional revulsion to them. The execution of attentional control and of cultivating strategically helpful emotion may be learned without conscious effort—may indeed be undermined by the awareness of conscious effort (Wegner, 1994)-- and in such cases fit the clinical meaning of repression and reaction formation respectively (Ainslie, 1989).

3.2. Beyond precommitment: Choice bundling as the basis of self-control

Tactics that commit choice in advance are sometimes evident in addicts’ efforts to avoid temptation. However, precommitment behaviors are not what people ordinarily have in mind when they refer to willpower or self-control; indeed, use of these expedients is usually seen as a sign that the person lacks confidence in her willpower, and thus has to resort to artificial means of self-control. People do not usually need to bind themselves by physical devices, contracts, or even reputation, to keep their intentions steady. It is certainly good advice for an addict to avoid the haunts where her substance is readily available; but most people who have given up a bad habit do not depend on keeping temptation at a distance or out of sight. People who have given up smoking, for instance, often say that one day they “just did it” (Premack, 1970). They are said to have used willpower.

The partially conflicting preferences of the self over time produced by hyperbolic discounting provide a bottom up account of willpower, in contrast to the holistic, top-down account that is conventional among cognitive theorists (Ainslie, 2006). Writers since antiquity have related willpower to choosing according to principle; that is, choosing in categories containing a number of expectable choices rather than just the choice at hand. Aristotle said that impulsive choice (“akrasia”) was the result of choosing according to “particulars” instead of “universals” (Aristotle, 1984, 24–28); Kant said that the highest kind of decision-making involved making all choices as if they defined universal rules (the “categorical imperative,”; (Kant, 1793/1960, 15–49); the Victorian psychologist Sully said that will consists of uniting “particular actions... under a common rule” so that “they are viewed as members of a class of actions subserving one comprehensive end” (Sully, 1884). The fundamental insight is that you increase your self-control by choosing according to category rather than on a case-by-case basis. But why should it do any good just to recognize that a given choice is a member of a larger category?

3.2.1. The effectiveness of reward bundling

Hyperbolic discounting predicts that the bundling of future smaller-sooner vs. larger-later choices with a current single smaller-sooner vs. larger-later choice diminishes impulsivity in that choice. Recall that, unlike the dynamic inconsistency implied by an exponential discount rate, hyperbolic discounting predicts that preference between a reward pair can shift as a function of the individual’s proximity in time to the rewards. When the individual is distant in time from the smaller-sooner reward, the larger-later is more valued, but when she is close in time to the smaller-sooner reward, the smaller-sooner is more valued (Figure 1b). If future choices of smaller-sooner vs. larger-later alternatives were bundled with present choices-- e.g., if the choice of whether to get drunk now at the expense of feeling good tomorrow entailed committing to the same alternative for every day for the next week-- the total value of the larger-later bundle relative to the smaller sooner bundle would increase, since the discounted value of the larger-later alternative in each future choice is greater than that of the smaller-sooner. Assuming that the discounted value of series of rewards is additive (as appears to be the case; Mazur, 1997), then bundling future identical choices with a current choice can reduce impulsive choice, given hyperbolic delay discounting but not exponential delay discounting (Figure 2a and 2b).¹

Figure 2.

Summed exponential (2A) and hyperbolic (2B) curves from a series of larger-later rewards and a series of smaller-earlier alternatives (vertical dashed lines). Each curve depicts the summed discounted values of all future (more to the right) rewards in the series. Thus the curves depicted at choice pair 1 are the sum of discounted value of the corresponding alternative in pairs 1 through 6, and the curves of choice pair 2 are the sum of 2 through 6, etc. For the hyperbolic (2B) but not for the exponential (2A) discount curve, as the series gets longer and the summed curves peak higher above the current rewards, the initial period of temporary preference (the period in which the SS curve is higher than its LL alternative) shrinks to zero.

Experiments with both human and rodent subjects confirm a greater tolerance for delay with bundled rewards. Kirby and Guastello (2001) gave college students choices between smaller and earlier rewards and larger but more delayed alternatives, both with money and food rewards. In one condition, the choice was made five times, each time separated by a week. In another condition, the choice was made between the two alternatives up front for all five weeks at once. As predicted from the summation of hyperbolically discounted rewards, preference for the later larger alternative was increased in the condition in which a series of choices was bundled together. Indeed, when Kirby and Guastello merely suggested to student subjects that the subjects’ current choices might serve as predictions of their future choices, preference for larger-later alternatives increased, although not as much as when the experimenters bundled the choices directly (Kirby and Guastello, 2001).

Human subjects are admittedly prone to both sensing an experimenter’s theories and implementing their own. However, the same phenomenon of decreased impulsive choice with bundling has been demonstrated in rats (Ainslie and Monterosso, 2003b). Eight rats were run through two conditions of a procedure designed to determine how much immediate sugar water was equal in value to a delayed standard reward of 150 ml after a three second interval. In one condition, subjects made choices on a trial-by-trial basis, while in another (“bundled”) condition they made choices on only every third trial, determining the reward that would be delivered for the next three consecutive trials. As predicted by figure 2B, but not 2A, preference for the later larger alternative was greater for all subjects in the bundled condition.

3.2.2. Forming bundles by the perception of precedent

The above findings suggest that choosing in categories would decrease impulsiveness, if only people could commit themselves to do it. However, unlike the above experiments, a person would have to make the bundles herself. How can someone do this in binding fashion, and how would she discover this trick? First, it is clear that people feel as if their current choice influences their future ones. If this needs demonstration, consider a thought experiment that magically removes someone’s otherwise inevitable uncertainty about what she will choose in the future (Monterosso and Ainslie, 1999): A smoker is preparing to quit, but currently craves a cigarette. Suppose an angel whispers in her ear that it is a forgone conclusion that, regardless of what she does now, she is destined to smoke a pack a day from tomorrow on. Given this certainty, would she have any incentive to turn down the desired cigarette? Turning it down would seem pointless. What if the destiny revealed by the angel was instead that, again regardless of what she does now, she was destined to never smoke again from tomorrow on? Here, too, there seems to be little incentive left to turn down the cigarette -- It would be harmless. Fixing future smoking choices in either direction evidently makes smoking the dominant current choice. Only if future smoking is in doubt does a current abstention seem worth the effort. But why should fixing future smoking behavior make a difference to the choice at hand? There is no literal connection between current and future choices. The fact that fixing future behavior undermines the incentive to turn down a current temptation suggests that more than just one episode is in the balance with a current choice.²

But in real life, how does this contingency develop? An astute person is aware that preferences are volatile within domains in which there is struggle for self-control. One way to predict behavior in the face of a future temptation is to see what occurs with a similar temptation in the present. Consider a smoker whose preference for cigarettes is roughly described by Fig 1b. That is, while she prefers smoking in the immediate future (for simplicity, consider the immediate future the present month), she also prefers nonsmoking in the more distant future. If she is deciding in January on a plan for her entire smoking future, the dominant option is to choose to smoke in January, but plan to abstain in February and thereafter. But what happens in February? Without bundling, in February she will simply change her mind, since now the dominant plan in terms of discounted pay-offs is to smoke in February and quit from March on. But of course, the same reversal, left unchecked, would occur when March becomes the present, and so on.

By April she may see the pattern of unrealized plans. She may notice her preference to stop smoking in May looks just like last month’s preference to stop smoking in April. Her plan will ultimately lose credibility. She may think something like “If I break last month’s plan to not smoke in April, I am going to do the same next month with my current plan to not smoke in May.” The credibility of attaining abstinence from May and beyond will thus depend upon not smoking in April. The smoker who sees her situation in this way is left with the bundled options of 1) expecting to smoke from the present on, versus 2) expecting not to smoke from the present on. If the expectation of not smoking is preferred given such a conception, the product is a personal rule such as “I cannot smoke any cigarettes.” The expectation of smoking in the future is thereby tied to one’s own adherence, or more accurately, one’s own perceptions of her adherence, to her rule in the present.

Similarly, in the example of money, the choice used most often in discounting research, it is self-evident that someone who makes monetary trade-offs for a living (e.g., an investor) must have a rule to choose as if she discounted delayed money at the current exponential market rate. For such a person, the choice made in a research study between $8 now versus $10 in a week may not be a choice between just the two alternatives; her expectation that she will follow her own financial rules may also be at stake, and with it, perhaps, her expectation of future financial prosperity generally. It may well be that if the contingencies were confined to the immediate $8 versus $10 in a week, that the former would seem the more valuable. But given the added stake she brings into the situation, the choice of the delayed $10 may have greater value than the choice of the delayed $8, because she experiences the value related to her own perception of her adherence to her rule. This leads to a plausible reinterpretation of the well-replicated finding that substance abusers discount money more steeply than comparison groups: Since a great many substance abusers have a history of squandering their money, the stake supporting any rules they may have had in the domain of money may have long been lost; substance abusers’ greater discounting might reflect the consequent weakness of their rules influencing financial decisions.

A person can thus reasonably interpret the intertemporal conflict between current and future selves as a variant of an iterated prisoner’s dilemma. In a conventional two- (or n-) person prisoner’s dilemma, the immediate opportunity cost associated with not defecting is offset by the perceived likelihood that a player’s counterparts will cooperate in future rounds if and only if the player cooperates in the present round. In the intertemporal variant, a future self will not be motivated to retaliate per se against a defecting present self,³ but will still be apt to defect because the pattern of cooperation that promised the bundled benefits has been broken. In both cases, the individual can be expected to cooperate only insofar as she sees cooperation as sufficiently necessary, and sufficiently effective at inducing cooperation (in either the opponent or future self). Just as people engaged in interpersonal prisoner’s dilemmas long before they were described as such, the intertemporal variant also goes unrecognized, even by people who play it skillfully.

Intertemporal bargaining is hard to study because it is intrinsically private and usually tacit. However, the similarity in contingencies can to some extent make the interpersonal bargaining situation a model for intertemporal bargaining. In one study, subjects played long strings of sequential prisoner’s dilemmas (Monterosso et al., 2002). When cooperation or defection became stable, false feedback was given to subjects indicating that their counterpart had broken the trend. The resulting responses displayed a large asymmetry, with false defection damaging mutual cooperation far more than false cooperation repaired mutual defection. Although original levels of mutual defection were restored after a single “recovery” move, cooperation rates were incompletely restored after even seven rounds of recovery moves following a single defection. While this finding does not confirm the role of analogous intertemporal bargaining in self-control, it is consistent with the lore on willpower (e.g. “every gain on the wrong side undoes the effect of many conquests on the right; Bain 1886: 440) and may be analogous to the “abstinence violation effect” (Marlatt and Gordon, 1980), in which any perceived slip in an area of self-control has a high likelihood of resulting in a binge of failed self-restraint. This effect has been documented in such disparate areas as drinking among alcoholics in treatment (Collins and Lapp, 1991), smoking among individuals attempting to quit (Shiffman et al., 1997, Spanier et al., 1996), eating among dieters (Grilo and Shiffman, 1994, Johnson et al., 1995), and fantasies among pedophiles (Hudson et al., 1992, Ward et al., 1994).

3.3 Abstention no longer dependent on distance from temptation

Hyperbolic mechanisms for addictive choices are sometimes criticized for seeming to force the choice into a Ulysses-and-the-Sirens mold. Clinicians are well aware that the precipitant for a relapse is not always, or even usually, sudden proximity with a tempting opportunity. Lapses sometimes follow a significant event, good or bad, in the person’s day, and are explained by the person herself as a one time celebration or consolation. Often they occur just when an excuse becomes handy.

Although this pattern would not be seen in a subject trying to control temptation by prior commitment, it is exactly what we would expect in someone struggling to use the willpower mechanism we have just described. A person’s intuitive understanding of the relevant intertemporal prisoner’s dilemma gives her the opportunity never to prefer small early alternatives at the expense of the series of larger later ones. When a whole bundle of later, later rewards is at stake she may be able to keep temptations close at hand without succumbing to them. However, although she may always prefer a series of larger later rewards to the small early one at hand, she must even more strongly prefer to have both.

The primary danger for this self-aware person is no longer one of the poorer reward coming so close that she will suddenly choose it. Instead, to the extent that her abstinence is based on a bundling effect, the primary danger comes from factors that reduce her differential expectation of future abstinence as a function of current abstinence⁴ Such a reduction can result variously from increased confidence that she can abstain in future similar situations, even if she indulges presently (overconfidence), reduced confidence that future abstinence will be achieved even if she presently abstains (underconfidence), or from the perception that a current opportunity to indulge is sufficiently dissimilar to ordinary situations so as to not inform her expectations about her future indulgence (rationalization).

It is not usually the case that the individual seeking treatment for substance dependence has just failed to discover willpower as the solution to her problem. She typically has called on willpower repeatedly, making resolutions that may have lasted only hours or days before failing. Explicitly teaching an addicted individual about the mechanism of willpower, of which they no doubt have intuitive understanding, will probably be useless. Significantly, the various schools of psychotherapy rarely try to make the will maneuver more forceful, but rather target people’s overzealous use of it—the “punitive superego” and its synonyms (Ainslie, 2001, pp. 143–155). An approach to addictions that does seem to use the properties of intertemporal bargaining, while deftly avoiding its pitfalls, is not seen in professional therapies at all, but in grass root “twelve-step” programs.

4. Intertemporal bargaining in 12-Step recovery

Twelve-step treatment for addiction appears to be, at least in some circumstances, more effective than an equivalent amount of either cognitive behavioral therapy or supportive expressive therapy (Crits-Christoph et al., 1999). Interestingly, evidence suggests that the advantage of 12-Step treatment (in terms of abstinence rates) over other therapies is most apparent during periods in which particularly high craving is experienced (Weiss et al., 2003). In other words, individuals in 12-step seem to do relatively well maintaining abstinence in the face of craving, ordinarily the job of willpower; but 12-step programs proclaim that willpower is not only unnecessary, but ineffective. We hold that they are correct about willpower as it is usually practiced, but that the strategies of these programs nevertheless depend on properties of intertemporal bargaining and thus invoke a close variant of willpower. In particular, we propose that aspects of the 12-step approach are responses to how overconfidence, underconfidence, and rationalization, as we have just defined them, pose primary threats to the maintenance of abstinence.

The declaration of powerlessness (Step 1: “We admitted we were powerless over our addiction - that our lives had become unmanageable”) may seem an unpromising starting point for a program designed to buttress self-control. However, among other things, this declaration of powerless effectively wards against overconfidence in the early stages of drug use cessation, by dismissing the often evidence-resistant notion that the individual can say “yes” now to the first drink, and say “no” later to the second, or tenth. It deters attempts to use rationalization and other hedges on willpower. “One drink is a thousand drinks” shores up one side of the critical perceived differential in conditional probabilities, by asserting that the probability of saying “no” later if you say “yes” now is zero. The threat of overconfidence that may develop later in recovery is also appreciated in the otherwise inexplicably dismal saying “every day brings you one day closer to your next relapse” as well as through the principle that addiction is a permanent condition, regardless of how long abstinence is maintained.

If self-control depends on present behavior informing expectation about future behavior within the whole category, then self-control should be inoperative when a temptation is not seen as belonging to the larger category. If the person who has vowed to stop smoking cigarettes does not perceive an opportunity to smoke a cigarillo as belonging to the same category, then there is nothing larger at stake in her response to the possibility. The problem is, of course, that since the payoff of smoke today and not smoke from tomorrow on is, ex hypothesi, higher than “abstain always,” she has incentive to rationalize individual cases as exceptions. As William James (James, 1890, pg 565) famously put it:

How many excuses does the drunkard find when each new temptation comes! It is a new brand of liquor which the interests of intellectual culture in such matter obliges him to test; moreover, it is poured out and it is a sin to waste it; or others are drinking and it would be churlishness to refuse; or it is but to enable him to sleep, or just to get through this job of work; or it isn’t drinking, it is because he feels so cold; or it is Christmas day; or it is a means of stimulating him to make a more powerful resolution in favor of abstinence than any he has hitherto made; or it is just this once, and once doesn’t count, etc., etc., ad libitum – it is, in fact, anything you like except being a drunkard.

Minimizing such rationalization is a major focus of 12-step recovery, as typified in the prayer for “freedom from self-will, rationalization and wishful thinking”. Twelve-step proponents adhere to the idea that total abstinence is the only possibility; there can be no exceptions or holidays from abstinence. If the alcoholic is helpless against alcohol, any lapse becomes a sign that the disease is going to take over. Instead of being nullified, the will is given the largest possible stake—all remaining hope of sobriety—but at the cost of no longer having any scope to redefine its terms. It was to a similar perception of helplessness that Max Weber attributed the increase of believers’ self-control in the wake of Calvin’s doctrine of predestination (1904/1958, p. 115)—arguably another example of making the will not actually helpless but, rather, transcendent (Ainslie, 1992, pp 203–204).

While perceived helplessness might seem to promote under-confidence that future abstinence can be maintained, the 12-step method fuels expectation that abstinence can be maintained. The ambitious resolutions that have ceased to be credible (“I’ll never drink again”) are replaced by believable building blocks: “one day at a time.” The believable expectation of one day’s sobriety becomes worth more than devalued long-range expectations—and yet the effect of a series of successful single days builds that very credibility that was lost, and this rebuilding is concretized in the practice of keeping a running total of how many days abstinent the participant “has.” The question of how this initially small stake is enough to motivate abstinence might be answered by the second step, “Come to believe that a Power greater than ourselves could restore us to sanity.” One of us has argued that putting at stake one’s good relationship with a felt other—guardian angel, god, observing ancestor, etc.—may provide much of the power of a personal rule with less risk of the legalistic rigidity that can seriously limit the effectiveness of intertemporal bargains (Ainslie, 2004).

5. The status of hyperbolic discounting as a factor in addiction and recovery

The theory of hyperbolic discounting implies a need for radical changes in RCT. It has been reasonable for authors to question whether these changes are necessary. This discussion has been ongoing in a number of areas (Ainslie, 2005a and its commentaries), but two suggestions seem important to discuss here: doubt stemming from the great variability of discounting rates in humans, and the possible adequacy of classical conditioning as an alternative mechanism for the pathogenesis and treatment of impulsive motives.

5.1 The variability of impatience

Behavioral economists’ response to the initial reports of hyperbolic discounting was to develop ways to measure a subject’s impatience factor (K in formula 2). Exponential discounting provided the natural point of comparison, and laboratory studies consistently observed behavior that was better fit by hyperbolic discount functions (Bickel et al., 1999, Madden et al., 1999, Vuchinich and Simpson, 1998). However, it did not take long to discover that there is no single K that can represent a subject’s basic discount rate. (Ainslie and Monterosso, 2003a, Loewenstein and Prelec, 1992, Loewenstein and Thaler, 1989, Roelofsma, 1996). Of most interest to addiction researchers, human delay discounting varies markedly from one domain to another in the same individual, so that, for example, an individual’s level of discounting money scarcely predicts their level of discounting of health outcomes (Chapman, 1996). This obviously undercuts the idea that estimates of delay discounting derived from a monetary choice procedure (or any other single reward procedure) provide a general index of impulsivity that can be used to predict behavior in other domains.

This variability, among other anomalies, has led some observers to question the usefulness of the temporal discounting approach:

“There is considerable doubt whether the psychological processes underlying [intertemporal choice] actually draw on a personal discount function... Decision makers appear to have as many discount rates as choice situations into which they can be placed. Moreover, different measures of discount rates are either uncorrelated, or are correlated weakly or idiosyncratically” (Roelofsma and Read, 2000, pg 171–172).

However, one source of this variability may be the bundling phenomenon described above: In humans the contingencies perceived to be at stake go well beyond the outcome of the choice at hand, and this is true for many different kinds of reward. Just as the intertemporal bargaining process may lead to the formation of rules about alcohol consumption, it can lead to tacit or explicit rules about monetary trade-offs between delay and amount, choices between short term and long term personal relationships, and even judgments about whether an impulse is consistent with the person’s perception of her character, the kind of consideration that Bodner & Prelec have called “self-signaling (2001; see Ainslie, 2005b). When a person structures her choices with personal rules she can be expected to express different preferences than she would if she were making a choice just on the basis of its own merits, and these preferences are apt to differ as well among categories of reward, according to their temporal distribution, emotional relevance, dangerousness, impulse control history, and doubtless many other factors. This difference may be responsible for some of the variability in delay intolerance that has been observed both between and within subjects.

The Ks derived from all human choices are apt to be reduced from what their spontaneous level would be by the greater or lesser influence of personal rules. Rules that emerge from intertemporal bargaining processes are domain specific – an individual’s experiences may lead her to be extremely rule bound in how she handles money, for instance, but spontaneous in how she manages her time. Variability in delay discounting, whether it be between individuals or between domains within the same individual, might relate less to differences in basic discounting than to differences in operative self-control processes (see Ainslie and Monterosso 2003). It may prove possible to get closer to the Ks for human subjects’ spontaneous preferences by using distraction or other techniques to make subjects less conscious of their choices; but until then, Ks should be interpreted as reflecting outcomes of the interaction of impulses and controls.

5.2 Can conditioning do the job?

A more robust objection to hyperbolic discounting theory is that the classical conditioning of appetite is an adequate explanation for sudden reversals of preference, and that conditioning might indeed be necessary to account for those reversals that do not follow signals of the imminent availability of consumption. People who are trying to avoid a temptation often lapse when the temptation is nearby, but also when they encounter a random reminder that leads to “irresistible” craving. Accordingly, the conventional explanation of short-sighted behaviors has been that they are motivated by classically conditioned appetites that are triggered by fortuitous stimuli, suddenly increasing the expected reward for a smaller, sooner alternative. This explanation has the virtue of being consistent with RCT; but classical conditioning does not withstand scrutiny as a selective mechanism independent of reward (for multiple reasons; see Ainslie, 1992, pp. 39–48, Ainslie, 2001, pp. 19–22), and conditioning is unnecessary to account for sudden craving, as we will now argue. It is interesting also, but only suggestive, that etiologies based on conditioning promise treatments using extinction or counterconditioning, but attempts to extinguish conditioned responses have been shown in a recent meta-analysis to be useless in recovery from addiction (Conklin and Tiffany, 2002).

Hyperbolic discounting offers solutions to the problems of classical conditioning theory through the hypothesis that classical conditioning is not a separate process from reward-based learning (Ainslie, 2001, pp. 48–70). The arousal of appetite, in this account, is an interest that competes with alternatives such as making money or listening to music in the common marketplace of expected reward. Appetites both increase the rewardingness of the events for which they prepare and produce some reward in their own right.⁵ Their occurrence thus will be mediated in part by the likelihood that the object of the appetite will follow. An increase will occur when consumption is imminent, of course, but it may also occur when someone thinks of consumption spontaneously. This will be true especially if the arousal of appetite makes consumption more probable or proximate; but insofar as an appetite is rewarding in its own right, it will depend less on the occurrence of its object for payoff, and may occur speculatively, as it were, merely occasioned by reminders of consumption.

If appetite is a process that makes consumption more rewarding, and occurs in part as a function of whether consumption is expected, then it has the potential to be a recursive phenomenon: When someone has chosen not to consume a given good, but experiences an increase in appetite, then if her resolve is not firm she may notice an increase in the probability that she will consume. But this increase may in turn increase her appetite, and so on. Someone who is confident of firm resolve would not be expected to experience the surge in appetite that results from this positive feedback loop (a prediction that is consistent with reports of the absence of craving among orthodox Jewish smokers on the Sabbath (Dar et al., 2005), and someone who expected to consume from the outset would feel appetite increasing as a smooth function of proximity; but people who are weakly resolved not to consume should be subject to this explosive surge of appetite. This is just the population in which the responses that are often described as cue conditioned are prominent (Maisto et al., 1977).

A more sophisticated alternative to hyperbolic discounting, still based on classical conditioning, explains dynamic inconsistency by a “hyperboloid” curve, which consists of a sudden drop in value as a reward is delayed by any amount whatsoever (“β component”), and a conventional exponential curve to describe the effect of all additional delay (Laibsen, 1997). The β component is explained as the “visceral” effect of either an immediate reward or of a classically conditioned stimulus (Loewenstein, 1996). This model, too is inspired by the common experience of having preference changed by a sudden surge of appetite, and it is supported by an fMRI finding that the striatal (β) reward center is active only when rewards might be relatively immediate; somewhat inconsistently, it casts immediacy as just “one of many factors that, by producing limbic activation [visceral motivation], engenders impatience” (McClure et al., 2004). This model holds particular attraction for economists, in that it preserves a realm--behavior with non-visceral motives-- in which standard economic tools requiring dynamically consistent preferences can be utilized.

The beta-delta/”visceral factors” proposal is contradicted by some other empirical findings, but there have still been few relevant fMRI data. Precise behavioral research on discounting in both human and nonhuman subjects reveals curves that do not simply demonstrate a spike in valuation at short delays, but rather appear to be discounted hyperbolically for their entire lengths (Green and Myerson, 2004), even when none of the choices is immediate (Green et al., 2005); and the striatal reward centers that were initially thought to be active only when prospective rewards were immediate (McClure et al., 2004) have been found to respond proportionally to delays (Glimcher and Kable, 2005, Monterosso et al., 2006), although the exact function by which this brain activity varies with delay has not been determined. The new possibility of fMRI data on brain activity associated with motivation opens an exciting chapter in addiction research, but it is too early to guess what it will say. So far its findings have neither contradicted nor supported the proposition that hyperbolic discounting is both necessary and sufficient for impulsive choice, and that classical conditioning is neither.

6. Can intertemporal bargaining be studied empirically?

It might be contended that intertemporal bargaining does not fit the framework of empirical science. As discussed above, the fundamental building block of the model is the recursive interplay between current preference and expectation of future behavior. What are the prospects of doing quantitative behavioral economic modeling of a hypothesized system with a recursive feedback loop between components that are not directly measurable?

With varying degrees of success, we, and others, have looked for ways to approach the topic empirically. Here we have briefly described demonstrations of decreased impulsiveness through choice-bundling in humans (Kirby & Guastello, 2001) and rodents (Ainslie and Monterosso, 2003b), modeling intertemporal bargaining with observable interpersonal bargaining (Monterosso et al., 2002), and clarifying common intuition with thought experiments (Monterosso & Ainslie, 1999; Ainslie, in press). There is also some value in testing different theories against the vast historical literature on will (Ainslie, 2001, pp. 117–120); but more direct evidence will probably depend on in vivo brain imaging. So far the fMRI data have been rudimentary, but technical developments (e.g., London et al., 2006) should permit replication with imaging of the behavioral experiments that have been done, and perhaps eventually visualization of the interaction of brain centers. Difficulty studying a phenomenon should not be a reason for discounting its importance. Whatever the methodological obstacles, we believe that intertemporal bargaining offers the best framework for scientific progress in the understanding of recovery from addiction.

Figure 3.

Depicted is the amount of sucrose solution that was equally preferred (selected 50% of the time) to a fixed standard of 150 ml sucrose solution delayed by 3 sec. As predicted by hyperbolic discounting, indifference points were higher (less discounting) in the condition during which each response determined the reward for three consecutive trials than in the standard condition during which each choice determined a single reward.