Gender differences in preference for reward frequency versus reward magnitude in decision-making under uncertainty

Abstract

Extensive research has focused on gender differences in intertemporal choices made from description in which participants must choose from multiple options that are specified without ambiguity. However, there has been limited work examining gender differences in intertemporal choices made from experience in which the possible payoffs among choice alternatives are not initially known and can only be gained from experience. Other work suggests that females attend more to reward frequency, whereas males attend more to reward magnitude. However, the tasks used in this research have been complex and did not examine intertemporal decision-making. To specifically test whether females are more sensitive to reward frequency and males are more sensitive to reward magnitude on intertemporal decisions made from experience, we designed a simple choice task in which participants pressed a response button at a time of their own choosing on each of many trials. Faster responses led to smaller, but more frequent rewards, whereas slower responses led to larger, but less frequently given rewards. As predicted, females tended to respond quicker for more certain, smaller rewards than males, supporting our prediction that women attend more to reward frequency whereas men attend more to reward magnitude.

1. Introduction

Decision-making is a complex process that is often surrounded by varying levels of risk and uncertainty. Given the significance of decision-making and the far-reaching consequences decisions can have, it is critical to understand how people make decisions and how individual difference factors affect decision-making strategies. Considerable work has focused on gender differences in risk-taking and description-based intertemporal decision-making (e.g., Eckel & Grossman, 2008; Jianakoplos & Bernasek, 1998; Overman, 2004; Reavis & Overman, 2001; van den Bos, Homberg, & de Visser, 2013; Weafer & de Wit, 2014). Recent work also suggests that males may focus more on reward magnitude, or on seeking options with the highest possible payoffs (Byrne & Worthy, 2016). In contrast, females tend to focus more on reward frequency, or on seeking options that provide smaller, but more consistent rewards. However, there has been limited work aimed at identifying how gender differences may influence experience-based intertemporal decisions. Thus, the purpose of this study is to assess whether females are more sensitive to reward frequency and males are more sensitive to reward magnitude on a novel experience-based intertemporal decision-making paradigm.

One broad way to dichotomize decision-making situations is whether they involve making decisions from description or experience (Hertwig, Barron, Weber, & Erev, 2004; Johnson & Busemeyer, 2010). In decision-making from description, the risks and rewards associated with each option are provided before the individual must make a choice. For example, people might choose between an insurance policy that is very cheap, but has a high deductible of several thousand dollars, versus a policy that is more expensive, but has a lower deductible. The key point is that the relevant information needed to make the decision is explicitly described rather than learned. Conversely, in decision-making from experience, the risks and rewards associated with each option are unknown, and the individual must learn from experience which alternative is best. For example, two new restaurants open up nearby and residents must try them out and learn from experience which one has the better food and atmosphere.

There is now extensive evidence that males are more risk seeking than females when making decisions from description (e.g., Croson & Gneezy, 2009; Eckel & Grossman, 2008; Jianakoplos & Bernasek, 1998; Powell & Ansic, 1997). In the example above, males would be more likely to prefer the insurance policy that has a low premium, but a high deductible if an accident happens. Females would more likely prefer the higher premium in order to avoid the risk having to pay a high deductible. Increased risk taking has also been found in one decision from experience task, the Balloon Analog Risk Task (BART; Cross, Copping, & Campbell, 2011). These findings suggest that compared to females, males are more willing to tolerate increasing levels of risk in order to pursue a large magnitude reward (Cross et al., 2011).

In addition to differences in risk sensitivity, recent work examining decision-making from experience suggests that males exhibit greater sensitivity to reward magnitude than females, while females demonstrate more sensitivity to reward frequency (Byrne & Worthy, 2016; van den Bos et al., 2013). Other work from intertemporal choice, or delay discounting, tasks which assess preference for immediate versus delayed rewards suggests that greater reward sensitivity in males may account for steeper discounting for real rewards (Weafer & de Wit, 2014). A recent review suggests that gender differences in inter-temporal choice may critically depend on task demands (Weafer & de Wit, 2014). In particular, females discount future rewards more steeply than males in delay discounting tasks where the rewards are hypothetical (Beck & Triplett, 2009; Smith & Hantula, 2008). In contrast, when a real monetary bonus is offered, males discount more than females (Kirby & Maraković, 1995, 1996). This conclusion is consistent with other work showing that males have a tendency to maximize future rewards, while females are biased toward optimizing immediate rewards (Byrne & Worthy, 2015). Thus, males’ enhanced risk-taking tendencies may be attributed to increased reward motivation for large rewards.

Perhaps one of the most widely used paradigms to assess gender differences in decision-making is the Iowa Gambling Task (IGT) in which individuals must learn the immediate and long-term payoffs of their choices by exploring different options (Bechara, Damasio, & Damasio, 2000). Several studies have demonstrated that males typically select options that yield larger long-term rewards on the IGT compared to females (Byrne & Worthy, 2016; Overman, 2004; Overman, Boettcher, Watterson, & Walsh, 2011; Reavis & Overman, 2001; van den Bos et al., 2013). Similar findings have also been found on the Soochow Gambling task, a variant of the IGT where the optimal options provide small losses on 80% of trials, but large gains on 20% of trials, leading to net positive long-term values. In contrast, the inferior options are appealing because they provide small gains on 80% of trials, but large losses on the remaining 20% of trials, leading to net negative long-term values (SGT; Chiu et al., 2008; Byrne & Worthy, 2016). In addition to overall performance differences in gambling tasks like the IGT and SGT, computational modeling findings demonstrate that males and females differ substantially in their decision-making strategies. In particular, females focus on the frequency of gains and losses, preferring options with frequent gains that provide the smallest variability between gains and losses. In contrast, males tend to place more weight on options with high expected values and large long-term gains, rather than reward frequency (Byrne & Worthy, 2016).

While this work from the IGT and SGT is consistent with the hypothesis that males are more sensitive to reward magnitude and females are more sensitive to reward frequency, it is important to note that these gambling tasks are complex and involve factors besides reward frequency and magnitude. In particular, gender differences in these tasks could be attributed to sensitivity to gains versus losses rather than sensitivity to reward frequency or magnitude. The task also involves learning the rewards associated with four different options, and participants must also learn which options are objectively better than the others. Thus, there is more to the task than simply learning which actions lead to more frequent rewards and which lead to rewards larger in magnitude, and these factors could have contributed to the observed gender differences.

In the present work, we sought to test whether gender differences in sensitivity to reward frequency and magnitude exist in a simpler intertemporal choice task, directly designed for such a purpose. To this end, we designed the Experience-Based Probabilistic Intertemporal Choice (EPIC) to measure preferences for more frequent rewards versus less frequent rewards that are larger in magnitude. On each trial of the EPIC task, participants were asked to press a button that then dispensed between 10 and 100 points. The magnitude and probability of receiving a reward (i.e. more than zero points) were determined by how long the participant waited to make a response. If participants responded faster, they had a high probability of receiving a low-magnitude reward. Thus, faster decision timing minimizes the risk of not receiving a reward at all, and should be more appealing to females if they have a stronger preference than males for frequently receiving rewards. In contrast, a slower decision time led to a low probability of receiving a large-magnitude reward, thereby increasing the uncertainty of receiving a reward, but increasing the magnitude of the reward that could be received. This inverse relationship between reward probability and magnitude as a function of decision timing serves two distinct purposes. First, this design keeps expected values for all decisions constant, which eliminates the potential confound that participants are basing decisions on expected value information, rather than sensitivity to reward frequency versus magnitude. Thus, differences in performance cannot be attributed to differences in the ability to learn which option has a higher objective value. Secondly, this task involves only gains, and no losses, thereby ruling out sensitivity to gains versus losses as a possible cause for differences in behavior.

Because the EPIC task involves an intertemporal component, we also had participants complete the delay discounting questionnaire (DDQ; Richards, Zhang, Mitchell, & de Wit, 1999; Worthy, Byrne, & Fields, 2014). The EPIC task requires each participant to determine their preferred delay, and thus level of uncertainty, in being rewarded based on experience gained throughout the task. In contrast, the DDQ assesses each participants’ preferred length of rewards delay from discrete, and descriptive, choices. While the DDQ does not involve decision-making from experience because the reward amount, length of delay, and outcomes of the choices are known, both the DDQ and EPIC tasks gauge how decision-makers discount rewards as a function of the delay in receipt.

Given the previous work showing that males prefer decisions that maximize large, long-term rewards (Byrne & Worthy, 2015, 2016), we predicted that males would exhibit a slower average decision time than females on the EPIC task compared to females. Such a finding would support our assertion that males are biased toward alternatives that offer the highest possible rewards, while females are less concerned with reward magnitude and instead prefer options that provide consistent, albeit small, rewards.

2. Method

We conducted two experiments; Experiment 2 was a gender-controlled replication of Experiment 1 to ensure that the gender effects initially found were not due to unequal sample sizes. This is important as replicability and reproducibility have recently emerged as important goals in psychological science (Zwaan, Etz, Lucas, & Donnellan, in press). Replicating the study allows for increased confidence that the results are reliable and replicable. It also allows us to combine the data and analyze them together, which increases our statistical power. To ease our exposition, we present the Methods and Results of both experiments simultaneously.

2.1. Participants

In the present studies, we used data from the undergraduate student population at a large university. Partial course completion credit was given in exchange for study participation in both experiments. Overall, there were 99 total participants: 49 participants in Experiment 1 (M_age = 18.86; SD = 1.95; 33 females) and 50 in the gender-controlled Experiment 2 (M_age = 18.46; SD = 0.79; 25 females).

2.2. Apparatus and procedure

All participants read and signed an Institutional Review Board-Approved consent form before beginning the experiment. All materials were presented on a personal computer screen running MATLAB and the PsychToolbox (v2.5) programs. Participants in the first experiment completed the Experience-based Probabilistic Intertemporal Choice Task and a standard intertemporal choice task in a counterbalanced order. In the second experiment, with the first experiment showing no effect of order, each participant completed the intertemporal choice task first. The standard intertemporal choice task used in both experiments was identical to the task described by Worthy et al. (2014).

2.2.1. Experience-based probabilistic intertemporal choice (EPIC) task

Before beginning the experiment, participants were told that they would go through several trials of a decision-making task where they would have a chance to win 10–100 points based on the time they pressed a button. They were not given any information about how rewards were calculated, but instead had to learn from experience over repeated trials. On each trial of the EPIC task, participants were presented with a black screen with a single square, green button. Participants were instructed that they could press the response key (‘B’) at any time which would trigger the green button onscreen. Based on the timing of their response, participants could either win or lose on each trial Participants’ response time determined the amount of points they could possibly win (curved, ascending line in Fig. 2 representing Possible Reward) and the chance of winning that amount (dashed descending line in Fig. 2 representing Chance of Winning). On a “win” trial, participants were shown the amount of points they were awarded (Fig. 1a). In contrast, on a non-win trial, participants received 0 points and were also shown the amount of points that would have been awarded had they won based on the time they waited on that trial (Fig. 1b). Note that participants were thus made aware of foregone rewards, or the amount they would have received on non-win trials (Byrne & Worthy, 2013). This was done to facilitate participants’ learning about what rewards were available at different response times. On every trial, participants had a hidden time limit to respond of 9 s. Exceeding the nine second limit would result in no points being awarded and a “Too Slow” prompt onscreen when the button was eventually pressed.

Fig. 2.

Plot of the EPIC task reward structure depicting the relationship between possible rewards, chance of reward (as a percentage), and expected value.

Fig. 1.

a: Depiction of “win” screen shown to participants. b: Depiction of “non-win” screen shown to participants.

The amount of possible points won on each trial was directly tied to the time the button was pressed. The probability of receiving a reward decreased the longer participants waited to respond – from 100% to a 10% chance, while the amount of possible points awarded increased over time from 10 to 100 points. To calculate whether or not the participant would receive the rewards associated with the response time, a random number between 0 and 1 was generated in the background on each trial (not seen by participants). If this number was less than the probability number associated with the participants’ response time, it would result in a win trial and points would be dispensed.

The possible reward values given at any point were set so that the expected value (EV) of each choice, computed as the product of the possible reward (x) and the reward probability (p), equated to 10 points regardless of how long participants took to respond: EV(x) = xp. An illustration of this reward structure can be seen in Fig. 2. Over 100 trials, participants worked toward a goal of 950 points and were asked to accrue as many points as possible by the end of the experiment. The primary dependent variable of interest in this task was the time at which the participants pressed the button (ref: response time) to obtain the reward.

2.2.2. Delay discounting questionnaire (DDQ)

In this task, participants were repeatedly shown two cards with differing monetary values and lengths of delay, and participants were instructed that they should choose whichever card they prefer. They were told that these choices were hypothetical, but to try to answer as if they would actually be receiving the money. One card consistently offered $10 after one of five possible delay periods (1, 2, 30, 180, and 365 days). The other card offered immediate rewards with an adjusting reward schedule where the rewards began at $2 and increased to $10 by $0.50 increments on each subsequent trial. The order in which each delay period was shown to participants was randomized. This procedure allows for the derivation of an indifference point for each delay period which is the smallest amount of money an individual chose to receive immediately instead of the $10 offered after the delay.

Using the indifference points, we empirically calculated the area under the discounting curve (AUC), for each participant as a measure of how much they discount guaranteed future rewards (Myerson, Green, & Warusawitharana, 2001). Greater area under the curve indicates less discounting of delayed rewards, while smaller area under the curve indicates increased discounting of delayed rewards and a preference for smaller immediate rewards.

3. Results

We briefly present the findings for each of the two experiments, and then discuss in more detail the findings using the combined data from each experiment. The dependent variables of interest–response time and intertemporal choice task performance – showed no effect of task order and no significant differences were observed for these variables between experiments: t(97) = 1.311, p = .193; t(97) = 0.122, p = .903, respectively. For all analyses using gender as a variable, we coded females as 0 and males as 1.

3.1. Experiment 1

We first compared the mean response times by gender. Males (M_{response time} = 2.81, SD = 1.74) waited significantly longer to respond than females (M_{response time} = 1.74, SD = 1.09), t(47) = 2.63, p = .011. Additionally, to examine the relationship between decision-making and temporal discounting of rewards, we analyzed the association between average response time on the EPIC task and AUC on the DDQ. The result was positive, but fell short of significance r(47) = 0.250, p = .083. No significant differences were found between genders and their respective discounting AUCs, t(47) = 0.245, p = .808.

3.2. Experiment 2

We completed the same analyses for Experiment 2 as we did in Experiment 1. Similar to Experiment 1, we observed a significant effect of gender on EPIC task response time: t(48) = 2.313, p = .025. Males (M = 2.89, SD = 1.66) waited significantly longer to respond, on average, than females (M = 2.01, SD = 1.09). The association between average EPIC task response time and AUC for the delay discounting task was non-significant, r(48) = 0.129, p = .382. There was no effect of gender on in the delay discounting task, t(48) = 0.716, p = .478.

3.3. Combined analyses

No significant differences were found between experiments in: participant response time, t(97) = 1.311, p = .193; average reward sought, t(97) = 1.142, p = .256; or proportion of gain trials, t(97) = 1.455, p = .149. Having confirmed that overall behavior was similar across the two experiments we combined the data to perform more detailed analyses on the full data set.

The difference in mean response time by gender remained significant with the combined datasets, t(97) = 3.695, p < .001. We then examined whether the gender differences we observed in average response times led to differences in points earned on the task, and we found that there was indeed a significant difference in points earned, t(97) = 2.136, p = .035. Females (N = 58) earned an average of 958 points while males (N = 41) earned an average of 923 points. A comparison of the proportion of trials that participants were rewarded on showed that, on average, females were rewarded on 79.28% of trials compared to 69.34% for males, t(97) = 3.414, p = .001. In terms of discounting AUC, males and females had an average AUC of 0.472 and0.454, respectively. No significant differences were found between the DDQ AUC and gender t(97) = −0.314, p = .754, and the correlation of intertemporal choice task and EPIC task response time, r(97) = 0.187, p = .067.

To examine whether the effect of gender on response time differed over the course of the task we divided the data into ten 10-trial blocks and computed the average response time during each block for each participant. Fig. 2 shows the average response time for males and females during each block. A 2 (gender) × 10 (block) mixed ANOVA revealed a significant effect of gender, F(1,97) = 13.66, p = .001, η_p² = 0.123. There was also a significant effect of block, F(4.49,435.41) = 4.092, p = .002, η_p² = 0.040, as well as a significant block X gender interaction, F(4.49,435.41) = 2.60, p = .030, η_p² = 0.026. To identify the locus of the block X gender interaction, we examined the effect of block for males and females separately. For males there was a strong effect of block F(4.35,173.91) = 3.541, p = .007, η_p² = 0.081. For females, however, there was no effect of block, F(4.27, 243.28) = 1.01, p = .408. As can be seen in Fig. 3, females consistently responded quicker than males, and males tended to respond more quickly as the task progressed.

Fig. 3.

Graphical depiction of overall averages, by gender, of response time across trial blocks. Errors bars represent the standard error of the mean.

4. Discussion

In two experiments, we examined how participants behaved in a task in which faster responses led to smaller but more probable rewards, while slower responses led to larger but less probable rewards. In accordance with our hypothesis, and corroborating previous research (Byrne & Worthy, 2016), females responded quicker than males which indicated a preference for smaller, more frequent rewards. Conversely, males tended to make slower responses that led to larger, but less consistent rewards. This was true for roughly the first 60 trials and males eventually began responding sooner as the task progressed. The results of these experiments suggest that there is a distinct difference in how males and females approach decision-making from experience.

While we did not find strong gender differences in delay discounting when examining area under the curve on the DDQ task, the direction of this small effect was consistent with our second hypothesis that males would display a less steep discounting curve compared to females. Average discounting curves did not significantly correlate with the average EPIC task response time. This may be due to the mismatch in time frames between the two tasks. The DDQ has a hypothetical time span that ranges from immediate to one year, whereas the EPIC task has a much shorter time span of 0–9 s. In addition, the discounting behavior of guaranteed future rewards may not exactly translate to the discounting behavior of uncertain rewards in an experience-based decision-making paradigm like the EPIC task. The results suggest that the EPIC and DDQ tasks may measure separate, if overlapping, constructs. Whereas the DDQ measures preferences for immediate versus delayed rewards, the EPIC task appears to measure preferences for small, certain rewards versus large, uncertain rewards, with females preferring the former and males preferring the latter.

The gender differences observed on the EPIC task are similar to previous findings on gambling tasks like the IGT and SGT (Byrne & Worthy, 2016). While the EPIC task is similar to gambling tasks in that both involve decision-making from experience, the EPIC task is less complex. It was designed to directly test the degree to which individuals prefer smaller, more probable rewards over larger, less probable rewards. Males demonstrated a greater willingness to wait for the possibility of receiving larger rewards even if it meant receiving no reward on more trials. This behavior is remarkably consistent to the behavior we observed in the IGT and SGT even though those tasks involved choosing among multiple options, rather than choosing among multiple response times. In the SGT, in particular, males most preferred the option that led to small losses on 80% of trials, but gave the largest gains on the other 20% of trials. Females, by contrast, most preferred the option that gave small gains on 80% of trials, but gave the largest losses on 20% of trials (Byrne & Worthy, 2016). We observed similar behavior in the EPIC task in which males showed a greater willingness to frequently forego reward, if it meant that they would occasionally receive high-magnitude rewards.

Combined, these results suggest that females may place a special emphasis on reward frequency, while males may attend more to reward magnitude. Future work should further test this hypothesis, as there are multiple paradigms and decision-making contexts where attention to reward frequency versus reward magnitude are significant factors. Additionally, future work could examine biological or sociological explanations for these gender effects in reward-based decision-making tasks. While we are not experts on evolutionary determinants of behavior, it is possible that evolutionary forces created different preferences for risk and uncertainty between males and females. Males may have been rewarded more throughout their evolutionary history for seeking large rewards that occurred infrequently, such as a big kill after a long hunt, whereas evolutionary pressures may have shaped females to prefer rewards that were smaller, but more consistent. However, we acknowledge that this explanation is speculative, but fields such as evolutionary psychology may benefit from addressing whether evolutionary pressures led to gender differences in reward seeking behavior. Conversely, these differences could be attributed to different socialization of gender roles.

It is also important to consider how the preferences for reward frequency versus magnitude are related to the well-established finding that males are more tolerant of risk than females. In the EPIC task females could be seen as making quicker responses to avoid the risk of not receiving anything. Thus, the stronger preference for more frequent rewards could be due to reduced tolerance of risk or uncertainty in females. However, the causal direction between preference for consistent rewards and intolerance of risk is difficult to establish. It could also be the case that females’ preference for consistent rewards leads them to exhibit more risk-avoidant and uncertainty-avoidant behavior. Males’ preference for seeking the largest possible rewards could lead to them tolerate more risk and uncertainty in order to achieve their goal of eventually obtaining a large payoff. In other words, differences in how males and females attend to the rewarding aspects of their environment could affect how much risk they are willing to tolerate – with males willing to tolerate more risk for the primary goal of obtaining larger magnitude rewards. Future work should seek to disentangle tolerance of risk and uncertainty from bias toward reward frequency versus reward magnitude.

Finally, the dopamine reward prediction error hypothesis is probably the most prominent neurobiological model of reward-based decision-making (Glimcher, 2011). Prediction errors, the difference between observed and predicted reward, putatively guide learning of the expected value for each option, and prediction errors have been linked to dopamine activity in the ventral striatum. Within this framework, one possibility is that males are biologically hardwired to process the magnitude of reward prediction errors, while females may be more responsive to the frequency of positive prediction errors. A related question is whether these differences in frequency versus magnitude preferences for reward also extend to losses. Males may focus more on avoiding losses of large magnitude, whereas females may focus more on consistently avoiding losses. These empirical questions and others can be tested in future work in order to identify why males and females seem to focus on different aspects of reward when making decisions.