Emotional arousal and discount rate in intertemporal choice are reference-dependent

Abstract

Many decisions involve weighing immediate gratification against future consequences. In such intertemporal choices, people often choose smaller, immediate rewards over larger delayed rewards. It has been proposed that emotional responses to immediate rewards lead us to choose them at our long-term expense. Here we utilize an objective measure of emotional arousal – pupil dilation – to examine the role of emotion in these decisions. We show that emotional arousal responses, as well as choices, in intertemporal choice tasks are reference-dependent and reflect the decision-maker’s recent history of offers. Arousal increases when less predictable rewards are better than expected, whether those rewards are immediate or delayed. Furthermore, when immediate rewards are less predictable than delayed rewards, participants tend to be patient. When delayed rewards are less predictable, immediate rewards are preferred. Our findings suggest that we can encourage people to be more patient by changing the context in which intertemporal choices are made.

In many everyday situations, we must decide whether to enjoy smaller benefits now or to wait for larger rewards in the future; these choices are known as intertemporal decisions (Strotz, 1956; Laibson, 1997). For example, we might choose to spend money now or save it for retirement. An addict might choose to consume a drug now or to forego that drug and enjoy positive future health benefits. Although people vary widely in the degree to which they discount the value of future rewards (Peters & Buchel, 2011), most people prefer immediate rewards to rewards received after a delay, sometimes even when the delayed reward is larger. One potential explanation for this inclination is that immediate rewards evoke a greater emotional response than do delayed rewards. In fact, this is a dominant theory in the intertemporal choice literature, and it has been formalized in two-systems models, which posit that our preference for immediate rewards stems from “hot” emotional responses, while patience emerges from more deliberative, “cold” reasoning (Figner, MacKinley, Wilkening & Weber, 2009; McClure, Laibson, Loewenstein & Cohen, 2004; Laibson, 1997). Despite these claims, few studies measure emotional responses during these decisions, and we are unaware of any studies that have examined the role of emotional arousal during intertemporal choice. The goal of the current study is to use an objective measure of physiological arousal – pupil dilation – to assess the role of emotion during this decision process.

We approached this question from a behavioral economics perspective by computing each participant’s hyperbolic discount “rate” (a measure of impulsivity; Strotz, 1956; Mazur, 1987) and using this parameter to quantify the value of the options that the subject faced on each trial. We used linear regression to delineate which task variables predicted emotional arousal on any given trial. This is critical, because autonomic arousal has been shown to be correlated with many psychological variables, including novelty and cognitive load (Chatham, Frank & Munakata, 2009; Laeng, Sirois & Gredeback, 2012; Goldwater, 1972; Kahneman & Beatty, 1966; Hess & Polt, 1960). We controlled for these factors in order to isolate the influence of emotion. To control for cognitive load, we included variables related to cognitive control (e.g., reaction time) as regressors in our analyses. To control for novelty, we varied the task structure in three conditions, so that we could examine how the predictability of a reward interacted with its value to influence emotional arousal.

Many measures of physiological arousal have been employed in studies of emotional and mood state over the last few decades (Cacioppo, Berntson, Larsen, Poehlmann & Ito, 2000), perhaps the most prominent being the skin conductance response (SCR; Naqvi & Bechara, 2006). All of these techniques rely on quantifying an output of the sympathetic nervous system, a branch of the peripheral nervous system dedicated to regulating a suite of physiological processes ranging from sweating to gastric motility. One sympathetic response that has received intense scrutiny in recent years is pupil dilation (Hartmann & Fischer, 2014). Empirically, pupil dilation has been shown to increase in response to affective images, and it covaries tightly with SCR (Bradley, Miccoli, Escrig & Lang, 2008). Pupil dilation is beginning to be widely adopted for affective studies because, unlike SCR, it has a short time lag (Boucsein, 1992; Figner & Murphy, 2011), and it is less influenced by factors such as age and gender (Eisdorfer, Doerr & Follette, 1980). Therefore, we used pupil dilation as our measure of emotional arousal.

In our study, participants performed three intertemporal choice tasks with different choice sets while pupil diameter data were collected, so that we could examine the relationship between value and emotional arousal in these tasks.

Method

Participants

All participants completed Conditions 1, 2 and 3 (“Delay Vary”, “Immediate Vary” and “All Vary”, described below) on three consecutive days. The order in which participants performed these three conditions was counterbalanced. As is standard in experimental economic studies, participants were excluded if their functions could not be fit on any one of the days (i.e., if they chose either all immediate or all delayed rewards in any of the conditions). This potentially imposes a selection bias, which is unavoidable in model-based studies that require successful fits of the model (e.g., Tymula et al., 2012; Tymula, Rosenberg Belmaker, Ruderman, Glimcher & Levy, 2013). Each subject was given $10/hour in cash for participating in the study, as well as additional compensation based on their decisions in the tasks. We chose thirty participants as our target sample size, because past studies of pupil dilation have drawn conclusions from approximately 20–30 participants (e.g., Cavanagh, Wiecki, Kochar & Frank, 2014; Nassar et al., 2012). Approval was obtained from the University Committee on Activities Involving Human Subjects at New York University, and all participants signed a consent form before the experiment.

Intertemporal choice task

On each day, participants performed an intertemporal choice task, while pupillometric data were collected. Participants were presented with a choice screen simultaneously showing two options: a monetary reward available today (e.g., “$10 today”), and a monetary reward of larger magnitude available at a later date (e.g., “$20 in 30 days”). They had 6 seconds to make a button press, indicating which of the options they preferred. The order of the trials was randomized; likewise, the immediate and delayed reward options switched sides of the screen in a random manner. After the allotted 6 seconds, a fixation point appeared for 2.5 seconds, followed by an outcome screen (3 seconds), which showed the participants what they had just chosen. After a 5 second inter-trial interval, the next choice screen appeared. The timing was the same for all paradigms, but the choices themselves differed. The structure of the choices for Conditions 1, 2 and 3 are described below. All paradigms were programmed with E-Prime 2.0 Stimulus Presentation Software (Psychology Software Tools Inc., Sharpsburg, PA).

To increase the saliency and relevance of their choices and to render our experiment incentive-compatible, participants were told at the outset that one of their responses from each task would be randomly selected and that they would receive the amount they chose on that trial, at the delay specified. That is, if they chose the immediate reward on the randomly selected trial, they would receive the money in cash as additional compensation after the session; conversely, if they chose the larger, delayed reward, they would receive a debit card that would only be activated after the delay had elapsed (Akimbocard.com).

Condition 1: “Delay Vary”

In the Delay Vary condition, participants were presented with 120 trials in the intertemporal choice task (60 distinct trials presented 2× each). The choice screen showed two options: one was an immediate reward of $10, $20 or $30, and the other was a reward of a higher magnitude to be received after one of the delays [7 days, 30 days, 60 days, 100 days, 180 days]. Delayed reward magnitudes depended on the magnitude of the immediate reward. $10 today was presented with $11, $15, $20 and $30 after a delay, $20 today was paired with $22, $30, $40 and $60 after a delay, and $30 today was paired with $33, $45, $60 and $90 after a delay. In order to probe for variability in discount rates, each delayed reward magnitude was paired with each delay.

Condition 2: “Immediate Vary”

In this condition, there were only three possible delayed rewards: $45 in 30 days, $60 in 30 days, or $90 in 30 days. Each of these was presented with one of 20 immediately available rewards. $45 in 30 days was presented with every monetary reward in the range [$4 – $44] in increments of $2. $60 in 30 days was presented with every reward in the range [$18 – $56] in increments of $2. Finally, $90 in 30 days was presented with every reward in the range [$28 – $85], in increments of $3.

Condition 3: “All Vary”

The purpose of the All Vary condition was to equate the variability of the immediate and delayed rewards within the same paradigm. For half of the trials, one of the choice options was an immediate reward of $10 or $20. The other option was one of 15 delayed rewards, each available in 30 days. The $10 immediate reward was paired with every reward in the range [$11 – $25] in increments of $1. The $20 immediate reward was paired with every reward in the range [$24 – $38] in increments of $1. For the other half of the trials in the task, one of the options was $30 in 30 days or $45 in 30 days. The other option was one of 15 immediately available rewards. The $30 reward in 30 days was paired with every amount in the range [$13 – $27] in increments of $1. The $45 reward in 30 days was paired with every amount in the range [$16 – $44] in increments of $2. The task was designed so that a participant with a discount rate (k) of 0.01 would choose immediate rewards on exactly half of the trials, and delayed rewards on the other half of the trials (0.01 was determined to be the mean discount rate in our population in past studies). In this paradigm, all delayed rewards were made available in 30 days, so that the probability of the value of the delayed reward changing from trial-to-trial equaled the probability of the value of the immediate reward changing on every trial. Each delayed reward has two attributes (magnitude and delay), so for the variability in immediate reward values and delayed reward values to match, the delay attribute had to be held constant.

Pupil diameter data collection and analysis

To collect pupil dilation data, we used Eye Link 1000 eye tracking equipment (SR Research Ltd., Mississauga, Ontario, CA). We sampled pupil diameter at 250 Hz throughout the entire task. Participants rested their chins on a chinrest. They were asked to minimize blinking and to focus their eyes on the screen during the experiment. To quantify pupil dilation responses during each choice, the pupil diameter data were analyzed using in-house software for Matlab 7.11 (MathWorks, Natick, MA), which was based on methods that were previously described (Henckens, Hermans, Pu, Joels & Hernandez, 2009). Eye-blinks were categorized as pupil dilation changes that transpired too quickly to represent actual pupil dilation; they were removed using linear interpolation. We normalized the data to the 1-second pre-stimulus onset baseline and then averaged the pupil diameter in the 1–4 second window after the choice screen was presented. The first second of stimulus presentation was excluded from analysis, since pupil diameter constricts during this time in response to luminance change. All choice screens in the task were equated for mean luminance.

Choice data analysis

To quantify each individual’s temporal discounting rate, we fit their choices to the hyperbolic model of temporal discounting (Mazur, 1987; Green & Myerson, 2004; Kable & Glimcher, 2007) and determined the best-fitting discount parameter k:

$$\mathrm{\delta }{\mathrm{V}}_{\text{del}}=\mathrm{A}/(1+k\mathrm{D})$$

Where δV_del is the discounted value of the delayed reward, A is the amount of the delayed reward, D is the delay, and k is the parameter that represents the participant’s discount rate (higher k values correspond to more impatience). We used this parameter to calculate the discounted value of the delayed reward for every trial for every participant (Kable & Glimcher, 2007). We assumed a linear utility function (see Kable & Glimcher, 2007; Andreoni & Sprenger, 2012 for implications of this assumption). In all ordinary least squares regression analyses, we accounted for systematic between-subjects differences by clustering standard errors by subject. That is, we assumed that model errors within subject would be correlated, although following standard econometric practice, we obtained “cluster-robust” standard errors post-estimation prior to statistical inference (for more details, see Cameron & Miller, 2011; Rogers, 1993) without an ex ante model. All regression analyses and error clustering were performed using STATA statistical software (StataCorp, College Station, TX).

Self-Report Questionnaires

After the participants completed the task, they filled out 6 questionnaires: (1) Delay of Gratification questionnaire (DoG), (2) Barratt Impulsiveness Scale (BIS), (3) Beck Depression Inventory (BDI-II), (4) State and Trait Anxiety Inventory – Trait version (STAIT), (5) the Self-regulation questionnaire (SRQ) and (6) a demographic questionnaire. The DoG questionnaire measures the participants’ readiness to delay immediately available rewards in favor of waiting for rewards that are distant, but more valuable (Hoerger, Quirk & Weed, 2011). Higher scores indicate greater willingness-to-wait for future rewards. The BIS tests for real-world impulsive personality traits (Patton, Stanford & Barratt, 1995); higher BIS scores correspond to more impulsivity. The BDI-II is the most widely used scale to measure the severity of depression (Beck, Steer & Brown, 1996). We assessed trait anxiety, which is the chronic, general susceptibility to be anxious, using the STAIT (Bados, Gomez-Benito & Galaguer, 2010). The SRQ was developed to assess the self-regulatory process through self-report; higher scores on this measure provide evidence for increased behavioral self-control (Ibanez, Ruiperez, Moya, Marques & Ortet, 2005). The final questionnaire collected demographic information (e.g., age, gender, ethnicity), for lab records.

Results

Forty-five participants completed the study (28 F, 17 M; mean age = 23.44; SD = 6.23). Fifteen were excluded, leaving thirty participants in final analyses (19 F, 11 M; mean age = 21.83; SD = 3.22). Twelve participants were excluded because their discount rates could not be fit on one or more of the three days in which we studied their behavior. Typically, discount rates could not be fit when a participant selected either all immediate or all delayed rewards in any one of the three tasks, thus preventing us from accurately measuring his discount rate on that day. In our sample, we found that six participants selected all immediate rewards, and six participants selected all delayed rewards. Thus, exclusions were balanced across our sample. In addition, two participants were excluded due to unreliable eye tracking, and one was excluded because he did not complete all three sessions.

Physiological arousal results

For every trial for every subject, we calculated 1) the value of the immediate reward (V_imm) in dollars, 2) the discounted value of the delayed reward (δV_del), 3) the (hyperbolically discounted) value of whichever option (immediate or delayed) was chosen on that trial (δV_chosen) and 4) the (hyperbolically discounted) value of whichever option had the higher value on that trial (δV_highervalue). We entered these variables into the models described below, to see which could significantly and best explain pupil dilation data in each of our three conditions¹.

• Model 1.

  Pupil = β₀ + β₁*(δV_chosen) + ε

• Model 2.

  Pupil = β₀ + β₁*(δV_highervalue) + ε

• Model 3.

  Pupil = β₀ + β₁*(δV_del) + β₂*(V_imm) + ε

Condition 1: Delay Vary

In this condition, Models 1, 2 and 3 yielded null results (see Table 1). That is, pupil dilation response on any given trial could not be explained by the value of the option that was chosen, the value of the option with the higher subjective value, the immediate reward value or the discounted value of the delayed reward.

Table 1.

Model comparison for Conditions 1 (Delay Vary) and 2 (Immediate Vary).

Models predicting pupil diameter at choice for Condition 1 (Delay Vary) and Condition 2 (Immediate Vary)
Regressors (dependent variable = pupil)	Coefficient (β)	P-value	95% CI	AIC
Condition 1 (delayed rewards varied)
δVchosen	.0001659	0.225	[−.000108, .000439]	−7386.298
δVhighervalue	.0001806	0.177	[−.000086, .0004479]	−7386.727
Vimm δVdel	.0002054 .0000596	0.399 0.768	[−.0002858, .0006966] [−.0003499, .000469]	−7384.171
δVdel - E[δVdel] Vimm - E[Vimm]	.0003716** −.0000398	0.001 0.782	[.0001586, .0005846] [−.0003316 .000252]	−7396.42
Condition 2 (immediate rewards varied)
δVchosen	.0000598	0.713	[−.0001079, .0004397]	−7311.105
δVhighervalue	.0000623	0.700	[−.0000866, .0004479]	−7311.142
Vimm δVdel	.0001709 −.0000843	0.454 0.843	[−.0002858, .0006966] [−.0003499, .000469]	−7312.453
δVdel - E[δVdel] Vimm - E[Vimm]	−.0002744* .0002643**	0.030 0.008	[−.0005206 −.0000282] [.0000747 .0004539]	−7315.808

Dependent variable is relative pupil diameter during choice. In Condition 1 (Delay Vary), pupil dilation is significantly positively predicted by the discounted value of the delayed reward relative to the expected discounted value of the delayed reward (i.e., the discounted value of the delayed reward averaged over the previous 20 trials). In Condition 2 (Immediate Vary), pupil dilation is significantly positively predicted by the value of the immediate reward relative to the expected value of the immediate reward (i.e., the value of the immediate reward averaged over the previous 20 trials). V_imm = value of the immediate reward in dollars; δV_del = discounted value of delayed reward, δV_chosen = value of whichever option (immediate or delayed) was chosen on that trial; δV_highervalue = value of whichever option had the higher value on that trial; E[V_imm] = average of immediate reward values over the previous 20 trials; E[δV_del] = average of discounted delayed reward values over the previous 20 trials.

^*p < 0.05.

^**p<0.01.

We then considered that arousal responses might depend on expectations, based on recent history. Past studies have shown that pupil dilation correlates with uncertainty in the task environment (e.g., Nassar et al., 2012) and with novelty (e.g., Naber, Alvarez & Nakayama, 2013), both of which necessitate a representation of the recent history of reinforcement. Therefore, we calculated an expected value of the immediate reward (E[V_imm] = the average of immediate reward values over the previous 20 trials) and an expected value of the delayed reward (E[δV_del] = the average of discounted delayed reward values over the previous 20 trials)². We computed the difference between V_imm and E[V_imm], as well as the difference between δV_del and E[δV_del], and entered these difference regressors into a linear regression model:

• Model 4.

  Pupil = β₀ + β₁*(V_imm − E[V_imm]) + β₂*(δV_del − E[δV_del]) + ε

In the Delay Vary condition, pupil dilation was significantly predicted by the discounted value of the delayed reward on that trial relative to the expected value of the delayed reward (β = 0.00037, p = 0.001; Table 1). Therefore, when the immediate reward value is relatively stable, the pupil responds to how much better the delayed reward value is relative to the recent history of delayed reward values. This effect was replicated in an independent sample of subjects that completed only the Delay Vary condition (see Supplemental Materials).

At first glance, it might seem like the pupil is simply responding to novelty, since the delayed reward was more likely to be novel on any given trial in this condition. However, pupil dilation is predicted not by how different or novel the current delayed reward is from the offers on previous trials, but rather, by how much better the current delayed reward is than expected. If it were simply a novelty response, then we would also expect arousal to increase when the current reward was worse than expected, but this was not observed. Furthermore, our model is significant only when we include the subjective (discounted) value of the delayed reward in our model, which depends on the subject’s idiosyncratic discount rate. This is consistent with the idea that we are capturing a valence-specific arousal response that is incorporated into the value computation.

Condition 2: Immediate Vary

As in the Delay Vary condition, Models 1, 2 and 3 yielded null results in the Immediate Vary condition. Once again, we fit Model 4, in which we took into account the recent history of offers in the paradigm. In contrast to the Delay Vary condition, here we found that pupil dilation increased as the value of the immediate reward increased relative to its expected value (β = 0.00026, p = 0.008; Table 1). This physiological arousal result was replicated in an independent sample that completed only the Immediate Vary task (see Supplemental Materials). We also found a significant negative relationship between pupil dilation and the value of the delayed reward relative to its expected value (β = −0.00027, p = 0.03; Table 1). This suggests that perhaps the pupil is sensitive to the difference between the delayed reward value and immediate reward value on the current trial, relative to the expected difference. Since this result was not found in the Delay Vary condition, however, and since it was not replicated in an independent sample, it should be interpreted with caution.

In the Immediate Vary condition, the immediate reward is more likely to change from trial to trial, and is therefore more likely to be better than expected. Thus, pupil dilation increases when the immediate reward is better than expected. The choice structure seems to be critical for determining which reward value (immediate or delayed) influences the emotional arousal response.

Condition 3: All Vary

In the All Vary condition, Models 1, 2, 3 and 4 did not significantly predict pupil dilation responses during choice (Table 2). This may reflect the fact that both immediate and delayed reward values varied at the same rate. Therefore, it is unclear which reward value is more likely to influence the emotional arousal response in this context.

Table 2.

Model comparison for Condition 3 (All Vary).

Models predicting pupil diameter at choice for Condition 3 (All Vary)
Regressors (dependent variable = pupil)	Coefficient (β)	P-value	95% CI	AIC
δVchosen	.0000761	0.739	[−.0003859, .0005381]	−8190.2
δVhighervalue	.0000512	0.824	[−.0004161, .0005186]	−8190.025
Vimm δVdel	.0003922 −.0002907	0.230 0.547	[−.000137, .0005635] [−.0007856, .0006219]	−8192.947
δVdel - E[δVdel] Vimm - E[Vimm]	−.0001276 .0003823	0.698 0.104	[−.0007939 .0005387] [−.0000836 .0008482]	−8192.635

Dependent variable is pupil diameter at choice. None of our models significantly predicted pupil dilation in the All Vary condition. Vimm = value of the immediate reward in dollars; δV_del = discounted value of delayed reward, δV_chosen = value of whichever option (immediate or delayed) was chosen on that trial; δV_highervalue = value of whichever option had the higher value on that trial; E[V_imm] = average of immediate reward values over the previous 20 trials; E[δV_del] = average of discounted delayed reward values over the previous 20 trials.

Controlling for cognitive load and difficulty

Since pupil dilation has also been associated with cognitive load and reaction time (Kahneman & Beatty, 1966), we examined the relationship between choice difficulty and pupil dilation. Taking reaction time as a proxy for cognitive load, we found no relationship between reaction time and pupil in any of the paradigms (although there was a trend in the Delay Vary condition: β = −0.000009; p = 0.056; and a very weak trend in the All Vary condition: β = −0.000004; p = 0.11; in the Immediate Vary condition: β = −0.000003; p = 0.51). We also found no significant relationship between pupil diameter and the absolute difference in value between the two options presented (|δV_del − V_imm|; when this difference is higher, the choice is easier; in Delay Vary: β = 0.00026; p = 0.26; in Immediate Vary: β = −0.00007; p = 0.62; in All Vary: β = −0.000003; p = 0.99). Furthermore, all effects reported here remained significant when controlling for reaction time and the difference in value between the choice options. In the Delay Vary condition, the δV_del − E[δV_del] term remained significant even when adding reaction time (β = .0003588; p = 0.004) and choice difficulty (β = .0003967; p = 0.009) to the model. In the Immediate Vary condition, the V_imm − E[V_imm] term remained significant when adding these two variables into the model (RT: β = .0002834; p = 0.007; difficulty: β = .0002725; p = 0.009). These findings indicate that, in our study, pupil dilation responses are not associated with cognitive load or difficulty. Instead, the arousal responses that we are recording seem to be affective in nature, with a positive valence, and they depend on the recent history of options presented.

Choice behavior

Although we did not anticipate that choice behavior would differ between conditions, we found that there was an effect of condition on discount rate. We fit discount rates using the hyperbolic model separately for the Delay Vary, Immediate Vary and All Vary conditions. Since discount rates are not normally distributed, we log-transformed them before performing further analyses. A repeated-measures ANOVA revealed that overall participants had significantly higher log-transformed discount rates (i.e., were more impulsive) in the Delay Vary condition, than in the Immediate Vary condition and the All Vary condition; F_(2,58) = 6.955; p = 0.002; η_p²= 0.193; Figure 1). Post-hoc paired t-tests showed that participants’ log-transformed discount rates were significantly higher in Delay Vary than in Immediate Vary (t₂₉ = 3.483; p = 0.002; Cohen’s d = 0.518; effect size r = 0.25), and significantly higher in Delay Vary than in All Vary (t₂₉ = 2.212; p = 0.035; Cohen’s d = 0.361; effect size r = 0.178). There was no significant difference in discount rate between Immediate Vary and All Vary conditions (t₂₉ = 1.444; p = 0.159). This shows that participants were more likely to select the more stable option: the immediate reward in the Delay Vary condition, and the delayed reward in the Immediate Vary condition.

Fig. 1.

History of offers affects discounting. Average log-transformed discount rate parameters k for Condition 1/Delay Vary (delayed rewards were more variable than immediate rewards), Condition 2/Immediate Vary (immediate rewards were more variable than delayed rewards) and Condition 3/All Vary (both immediate and delayed rewards were equally unpredictable). Higher k values indicate more impulsivity; lower k values signify more patience. Individuals were significantly more impulsive in the Delay Vary condition than in the Immediate Vary and All Vary conditions. Error bars represent SEM. (N = 30; F_(2,58) = 6.955; p = 0.002; * p < 0.05; **p < 0.01, + p < 0.2).

Although discount rate varied across conditions, we wanted to determine if individual participants’ relative discount rates were stable from task to task. To this end, we ran a Spearman’s rank order correlation. We found that the rank orders of discount rates in the three conditions were significantly positively correlated with each other (Delay Vary and Immediate Vary, ρ = 0.754, p < 0.001; Delay Vary and All Vary, ρ = 0.752, p < 0.001; Immediate Vary and All Vary, ρ = 0.811, p < 0.001). Therefore, the most impulsive participants in any given paradigm were still the most impulsive in other paradigms (the same applied to the most patient participants), but the mean discount rate of the sample shifted from condition to condition.

We considered, however, that the effect of condition on discount rate may have been due to design differences between the different tasks that were unrelated to the variability of rewards. For example, while the delayed rewards varied in delay to receipt in the Delay Vary condition (7 days − 180 days), they were always available after 30 days in the Immediate Vary and All Vary conditions. Because of this, the range of potential temporal discount rates was larger in the Delay Vary condition, and this may have biased our results. Secondly, the potential delayed reward amounts were larger overall in the Immediate Vary condition. Therefore, it is possible that increased patience in this condition may have been due to the magnitude effect, in which discount rates decrease as potential future rewards increase in monetary value (Thaler, 1981). To verify that our behavioral result was due to variability and not these other variables, we conducted a follow-up behavioral study in which we controlled for these important confounding variables and found the same pattern of results (i.e., participants were most impulsive in the Delay Vary condition and most patient in the Immediate Vary condition; see Supplemental Materials). None of the questionnaire scores correlated with discount rate in any of the paradigms³.

Discussion

This investigation of emotional arousal during intertemporal choice uncovered two key results. First, emotional arousal responses track the value of the more variable reward relative to its expected value (i.e., the average of this reward value over the last 20 trials). Critically, pupil dilation increased as the rewards increased in value relative to the expected value, and not as they decreased in value, signaling that we are capturing a valenced affective response, and not simply a novelty response. This relationship is also robust to variations in difficulty throughout the task. Therefore, the more motivationally relevant stimulus (an unpredictable reward option of high value) will elicit a larger pupil dilation response, and this depends on the choice environment and the recent history of reward.

Our second main result is that when delayed rewards are more variable (i.e., more likely to change from trial-to-trial) than immediate rewards are, people are more impulsive, and when immediate rewards are more variable than delayed rewards are, people are more patient. This indicates that the recent history of offers may influence valuation in this paradigm. While the finding that intertemporal choices are reference-dependent is not novel (Loewenstein, 1988; Weber et al., 2007), our study demonstrates for the first time that reference-dependence in intertemporal choice can be induced by changes in expectations. This result has implications for policy. For example, institutions can encourage decision-makers to be more patient simply by keeping delayed rewards relatively constant while imposing greater variation on immediate rewards. It also suggests that discount rates reported in the literature may be valid only for certain conditions.

Despite the prominence of emotion in intertemporal choice theory, few studies have explored the role of emotion or physiological arousal in intertemporal choice. In contradiction to the hypothesis that immediate rewards are more emotionally salient than delayed rewards are (McClure et al., 2004; Laibson, 1997), we did not find that pupil dilation increased preferentially in response to immediate reward values. Based on our findings, it seems more likely that the role of emotional arousal in this choice process depends on the context. These emotional responses are then most likely incorporated into valuation during choice in a more integrated fashion, suggesting a modulatory role for emotional arousal during decision-making (Phelps, Lempert & Sokol-Hessner, 2014). Since our design was correlational in nature, however, we are unable to draw any conclusions about the relationship between reference-dependence in temporal discount rate and reference-dependence in arousal. One possibility is that participants are more likely to choose the more unchanging reward because it becomes their point of reference (or “default” option), but that the more unpredictable reward value is more salient. This reward may elicit an emotional response when it is larger than expected, thereby leading participants to switch from default responding. Future studies should explicitly manipulate arousal to determine if it underlies reference-dependence in intertemporal choices.

In our experiment, we used an objective measure of emotional arousal (pupil dilation), but no convergent subjective measures of emotion. Using an objective measure is advantageous because it does not require participants to consciously monitor their emotional responses during the task. Previous research has shown that introspecting on an emotional response may actually change the response (Hutcherson et al., 2005; Silvia, 2002). Few studies have collected subjective emotion ratings after each choice in an intertemporal choice paradigm. In one such study (Benoit, Gilbert & Burgess, 2010), emotional intensity ratings of delayed rewards increased when the individual engaged in concrete future-directed thinking. These ratings predicted more patient choices. The results of this study are consistent with ours, since it showed that delayed rewards can elicit emotional reactions that are linked to decisions, and these reactions can be modulated by context.

We collected self-report data regarding real-world impulsivity (BIS, DoG, SRQ) and emotional traits (STAI, BDI-II), but we did not find any significant relationship between these measures and discount rate in our paradigm. This provides further evidence that impulsivity is a multidimensional construct (Duckworth & Kern, 2011), and that self-report and financial discounting task measures may be capturing different aspects of impulsivity (Reynolds, Ortengren, Richards & De Wit, 2006). Indeed, the correlation between BIS scores and temporal discount rate is very weak in many studies (Lange & Eggert, 2014; Reynolds et al., 2006; Reynolds, Penfold & Patak, 2008; Lempert & Pizzagalli, 2010). The relationship between self-report measures of depression and anxiety and temporal discount rate has, however, been relatively understudied. There are mixed results concerning the relationship between depressive symptoms and temporal discounting (Jarmolowicz et al., 2014; but see also Lempert & Pizzagalli, 2010).

In conclusion, we found that physiological arousal responses during intertemporal choice are reference-dependent – they track information about the value of the less predictable reward in the environment. Furthermore, we have demonstrated that choices between immediate and delayed rewards are also dependent on the structure of the choice set. Just as our knowledge of the environment influences our physiological arousal responses, it also affects our decisions. Awareness of this flexibility is critical if we want to make consistently adaptive choices.