Suboptimal Choice by Pigeons: An Analog of Human Gambling Behavior

Abstract

Human gambling often involves the choice of a low probability but high valued outcome over a high probability (certain) low valued outcome (not gambling) that is economically more optimal. We have developed an analog of gambling in which pigeons prefer a suboptimal alternative that infrequently provides a signal for a high probability (or high magnitude) of reinforcement over an optimal alternative that always provides a signal for a lower probability (or lower magnitude) of reinforcement. We have identified two mechanisms that may be responsible for this suboptimal behavior. First, the effect of nonreinforcement results in considerably less inhibition of choice than ideally it should. Second, the frequency of the occurrence of the signal for a high probability or high magnitude of reinforcement is less important than ideally it should. Also analogous to human gambling is the finding that pigeons that are normally food restricted choose suboptimally, whereas those that are minimally food restricted choose optimally. In addition, pigeons that are singly housed choose suboptimally, whereas those that are exposed to a more enriched environment choose less suboptimally. We believe that these findings have implications for the understanding and treatment of problem gambling behavior.

1. Introduction

Problem gambling in humans is clinically recognized as an impulse control disorder in which people show impaired behavioral inhibition and a failure to consider the long-term consequences of the decisions they make (DSM-IV-TR; APA, 2000). When gambling is suboptimal, it refers to choices in which the average net return is less than what is wagered (most commercial gambling). Such gambles are typical of casino games such as slot machines, roulette, and black-jack, and are especially true of lotteries. Because the net return on such decisions is negative, the decision to choose to gamble and receive a low-probability, high-payoff outcome (losing most of the time and winning occasionally) rather than not to gamble and maintain a high-probability low-payoff outcome (the amount not wagered) is viewed as a failure to maximize gains and minimize losses.

One explanation for human gambling has to do with the fact that in most public gambling, when someone wins, it is more salient than when someone loses (bells ring and lights flash at casinos when someone wins big and big winners of lotteries are often mentioned on the news). This is sometimes referred to as an example of the availability heuristic (Tversky & Kahneman, 1974).

Interestingly, examination of the behavioral ecology literature suggests that one should not find evidence of suboptimal choice in nonhuman animals because natural selection should have selected animals to be optimal foragers (Stephens & Krebs, 1986). Given appropriate experience, nonhuman animals are presumed to be sensitive to the relative amounts of food obtained from different alternatives or patches (see Fantino & Abarca, 1985).

2. A rat model of human gambling

One gambling-like task that has been modified for use with animals is the Iowa Gambling Task (Rivalan, Ahmed, Dellu-Hagedorn, 2009; Zeeb, Robbins, & Winstanley, 2009). In the Zeeb et al. study, rats chose among four options that varied among them in the probability of reinforcement (0.4 to 0.9), amount of reinforcement (1-4 pellets), probability of a punishment timeout following a trial (0.1 to 0.6), and the duration of the timeout (5 s to 40 s). Using this task, Zeeb et al. found that the rats chose adaptively, maximizing food pellets earned per unit time.

Interestingly, the rats failed to choose optimally when the probability of the time out was varied, even though the longer timeout meant that it occurred less often per unit time. Under those conditions, they undervalued the negative effects of the long time outs and instead were attracted to the larger magnitude of reinforcement. This meant that in so doing they received only half of the maximum number of pellets per unit time.

Rivalan et al. (2009) gave rats a choice between one alternative that provided a small amount of food on some trials and a short penalty on other trials and a second alternative that provided a larger amount of food on some trials but a very long penalty on other trials. However, because of the long penalties, the alternative associated with the larger amount of food actually resulted in only 20% as much food per unit time. Although a majority of the rats performed optimally and chose the alternative that provided a small amount of food and the short penalty, a substantial number of the rats preferred the alternative that provided a larger amount of food and the longer penalty. These results suggest that some rats may be relatively insensitive to the duration of the penalty and thus perform suboptimally in terms of the amount of food obtained per session.

3. A pigeon model of human gambling

There is substantial evidence that pigeons prefer choices that produce discriminative stimuli over those that do not. Specifically, they prefer choices that sometimes result in a strong conditioned reinforcer (followed by reinforcement 100% of the time) and sometimes result in a strong conditioned inhibitor (never followed by reinforcement) over those that result in weak conditioned reinforcers (followed by reinforcement 50% of the time) even though choice of either alternative would result in the same amount of reinforcement (see Figure 1; Roper & Zentall, 1999).

Figure 1.

Procedure used in Roper and Zentall (1999). Pigeons chose between two alternatives. Choice of one alternative (e.g., left) was followed by either a stimulus (e.g., red) 50% of the time that was always followed by reinforcement or a different stimulus (e.g., green) 50% of the time that was never followed by reinforcement. Choice of the other alternative (i.e., right) was followed by either of two stimuli (blue or yellow) both of which were followed by reinforcement 50% of the time. Spatial location and colors were counterbalanced.

But would pigeons prefer an alternative that produced discriminative stimuli if it resulted in a significantly lower probability of reinforcement? Apparently they would. Under the right conditions, some pigeons prefer an alternative associated with 50% reinforcement that produces discriminative stimuli (half of the time a stimulus that reliably predicted reinforcement, half of the time a different stimulus that reliably predicted the absence of reinforcement) over an alternative that always predicts reinforcement (Belke & Spetch, 1994; Fantino, Dunn, & Meck, 1979; Mazur, 1996; Spetch, Belke, Barnet, Dunn, & Pierce, 1990; Spetch, Mondloch, Belke, & Dunn, 1994). Under these conditions, when given a choice between 50% reinforcement and 100% reinforcement, some pigeons choose the 50% reinforcement option (although others did not). In this case, both alternatives are associated with strong conditioned reinforcers. We will return to this condition in a later section.

In a more recent experiment, we attempted to get more consistent preferences while maintaining the lower probability of reinforcement associated with choice of the alternative followed by the discriminative stimuli (Gipson, Alessandri, Miller, & Zentall, 2009). In this experiment, we pitted 50% reinforcement with discriminative stimuli against 75% reinforcement with nondiscriminative stimuli (see the design in Figure 2). These pigeons were given a choice between two white lights, one on the left the other on the right. A single peck to one light resulted in the presentation of one of two colored lights (S1 or S2) for 30 s. If it had been S1, it was always followed by reinforcement. If it had been S2, it was never followed by reinforcement. Thus, choice of that alternative resulted in the appearance of a discriminative stimulus and the overall probability of reinforcement was 0.50. A single peck to the other white light resulted in the presentation of one of two different colored lights (S3 or S4) for 30 s and in either case it was followed by reinforcement with a probability of 0.75. Thus, choice of the second alternative resulted in a higher probability of reinforcement than choice of the first alternative. To ensure that the pigeons had adequate experience with the contingencies of reinforcement associated with the two alternatives, in each training session the pigeons received 12 forced trials with each discriminative and nondiscriminative terminal link stimulus and 12 choice trials. With this design we found a statistically reliable suboptimal preference of 69% for the alternative associated with 50% reinforcement.

Figure 2.

Procedure used in Gipson et al. (2009). Pigeons chose between two alternatives. Choice of one alternative (e.g., left) was followed by either a stimulus (e.g., red) that was always followed by reinforcement on half of the trials or a different stimulus (e.g., green) that was never followed by reinforcement on the remaining trials. Choice of the other alternative (i.e., right) was followed by either of two stimuli (blue or yellow) both of which were followed by reinforcement 75% of the time. Spatial location and colors were counterbalanced.

In a follow up study, we found that if we reduced the probability of reinforcement associated with the discriminative stimulus alternative, we could obtain an even larger preference for that alternative (Stagner & Zentall, 2010). Specifically, the probability of reinforcement associated with the discriminative stimulus alternative was only 0.20 (the stimulus that reliably predicted reinforcement occurred on only 20% of the trials), whereas the probability of reinforcement associated with the nondiscriminative stimulus alternative was 0.50 (2.5 times the probability reinforcement associated with the discriminative stimulus alternative, see Figure 3). Under these conditions, the pigeons showed an even stronger preference (97%) for the discriminative stimulus alternative.

Figure 3.

Procedure used in Stagner and Zentall (2010). Pigeons chose between two alternatives. Choice of one alternative (e.g., left) was followed either by a stimulus (e.g., red) on 20% of the trials that was always followed by reinforcement or by a different stimulus (e.g., green) on 80% of the trials that was never followed by reinforcement. Choice of the other alternative (i.e., right) was followed by either of two stimuli (blue or yellow) each of which was followed by reinforcement 50% of the time. Spatial location and colors were counterbalanced.

3.1 The value of the suboptimal choice: Reduced delay to reinforcement

It is well known that delay to reinforcement is a powerful variable that can cause organisms to choose suboptimally (Fantino & Abarca, 1982). ). For example, Rachlin and Green (1972) showed that pigeons preferred a small immediate reinforcer (2-s access to reinforcement) over a larger reinforcer (4-s access to reinforcement) delayed by 4 s. But not all suboptimal choices are irrational. People sometimes pay twice as much over time for a mortgage to be able to live in a house rather than delay living in the house for 20 years. Similarly, a hungry animal may not want to wait for a larger amount of food if it has the opportunity to eat immediately. In all of the experiments involving the present pigeon model of suboptimal choice, the time between the choice response and reinforcement (or its absence) is carefully controled by presenting the cues associated reinforcement or its absence for a fixed duration (independent of the pigeons’ response). However, one way to assess the value of the discriminative stimulus alternative is to ask how much shorter the delay to reinforcement must be following choice of the alternative associated with the nondiscriminative stimuli, for pigeons to shift their preference to that alternative.

In this experiment we used the procedure described by Mazur (1996) in which the delay to reinforcement for the two alternatives starts out the same but is gradually reduced for the nondiscriminative alternative until the preference switches and then the delay to reinforcement for the nondiscriminative alternative is gradually increased until it switches back to the discriminative stimulus alternative (Zentall & Stagner, 2011b, Exp. 1). The results of this experiment indicated that the duration of conditioned reinforcer associated with the optimal (50% reinforcement) alternative had to be reduced from 10.0 s to about 4.4 s before the pigeons began to prefer it over the suboptimal (20% reinforcement) alternative for which the conditioned reinforcer remained at 10.0 s.

3.2 The Allais paradox

Humans often show a paradoxical choice behavior sometimes referred to as the Allais paradox (Allais, 1953) or the certainty effect (Shafir, Reich, Tsur, Erev, & Lotem, 2008). For example, if humans are given a choice between a 100% chance of earning $5 or an 80% chance of earning $10, although the average return on the 80% chance of earning $10 is higher ($8), most people choose the certain $5. But paradoxically, if one reduces both of the probabilities by one half (i.e., a choice between a 50% chance of earning $5 and a 40% chance of earning $10), the opposite preference will typically be found. According to expected utility theory, the results of the second choice should be the same as the first choice but they are not. The reason subjects often give for the preference for the certain $5 is they would be especially disappointed if they chose the 80% chance of $10 and lost, whereas in the case of the preference for the 40% of obtaining $10 they reason that they could almost as easily have lost had they chosen the 50% chance of obtaining $5.

If humans choose suboptimally to avoid the possibility of a loss, could that also be why the pigeons choose the alternative that provides the conditioned reinforcer that predicts 100% reinforcement over the alternative that provides a conditioned reinforcer that predicts 50% reinforcement? To test this hypothesis we conducted an experiment similar to that of Stagner and Zentall (2010) in which the probabilities of reinforcement associated with the conditioned reinforcers were reduced by 20%. As in the Stagner and Zentall study, the probability of reinforcement associated with the discriminative stimulus alternative was only 0.20 (the stimulus that reliably predicted reinforcement occurred on only 20% of the trials), however, on those trials, reinforcement occurred only 80% of the time. Thus, reinforcement was no longer certain. To maintain the same ratio of reinforcement to the optimal alternative, the probability of reinforcement associated with the nondiscriminative stimulus alternative was reduced to 0.40. Thus, the probability of reinforcement associated with the discriminative stimulus alternative was now .16 whereas the probability of reinforcement associated with the nondiscriminative stimulus alternative was now .40 (again, a ratio of 1:2.5). Once again, however, the pigeons showed a strong preference for the discriminative stimulus alternative (Zentall & Stagner, 2011b). Thus, the uncertainty associated with the conditioned reinforcer that followed choice of the suboptimal alternative did not deter the pigeons from choosing suboptimally. Of course, it is possible that if the probability of reinforcement associated with low probability high payoff stimulus was reduced still further, for example, to .50 (i.e., to a 20% chance of receiving a stimulus associated with 50% reinforcement) and the probability of reinforcement associated with high probability low payoff stimuli were reduced accordingly (to a 100% chance of receiving a stimulus associated with 25% reinforcement) the pigeons’ choice would have reversed to become optimal. However, as we will see in the next section, certainty does not appear to be the mechanism responsible for suboptimal choice when probability of reinforcement is manipulated (Zentall & Stagner, 2011a).

4. A better pigeon analog of human gambling behavior

Although the results of experiments by Gipson et al. (2009) and Stagner and Zentall (2010) clearly demonstrated suboptimal choice behavior by pigeons, when humans gamble, the alternatives generally involve different magnitudes of reinforcement (typically money) rather than different probabilities of reinforcement. For example, one may purchase a lottery ticket for $1 in hope of winning a large amount of money. It is possible that the effect we have been observing with the manipulation of probability of reinforcement occurs because the pigeons are avoiding an alternative that results in stimuli associated with an uncertain outcome (0.75 probability of reinforcement in Gipson et al., 2009, and 0.50 probability of reinforcement in Stagner & Zentall 2010). If the effect that we have been studying with pigeons is a good analog of human gambling behavior, it should be possible to find a similar effect by manipulating the magnitude of reinforcement, rather than the probability of reinforcement, and removing the uncertainty of the outcome associated with the nondiscriminative stimuli.

Zentall and Stagner (2011a) gave pigeons a choice between two alternatives. Choice of one alternative on 20% of the trials produced a stimulus that always predicted the delivery of 10 pellets of food and on the remaining 80% of the trials, produced a stimulus that always predicted the delivery of 0 pellets. Thus, this alternative was associated with an average of 2 pellets per trial (see design in Figure 4). Choice of the other alternative always produced one of two stimuli each of which always predicted the delivery of 3 pellets. Thus, the second alternative was associated with a consistent 3 pellets per trial. Once again, if pigeons are sensitive to the amount of food they obtain over time, they should select the 3-pellet option. However, contrary to this prediction, the pigeons showed a strong preference for the variable 2-pellet alternative over the fixed 3-pellet alternative.

Figure 4.

Pigeons chose between two alternatives that were distinguished by discriminative stimuli (a vertical or a horizontal line). Choice of one alternative was followed either by a stimulus (e.g., red) on 20% of the trials that was always followed by 10 pellets of reinforcement or by a different stimulus (e.g., green) on 80% of the trials that was never followed by reinforcement. Choice of the other alternative was followed by either of two stimuli (blue or yellow) both of which always were followed by 3 pellets of reinforcement. Spatial location and colors were counterbalanced.

However, preference for the 2-pellet option could also be interpreted as a preference for the variable option (10 pellets 20% of the time, 0 pellets 80% of the time), whereas the 3-pellet option was a constant 3 pellets, and it is well known variable schedules are preferred over fixed schedules. For example, Fantino (1967; see also Hursh & Fantino, 1975) found that pigeons preferred a mixed fixed ratio (FR)25 FR75 schedule over a constant FR50 and the preference was even greater for a mixed FR10 FR90 schedule. Fantino concluded that the relative preference was best described by the geometric mean. Thus, the FR50 would have been compared to the geometric mean for the mixed FR25 FR75 which was 43.3, whereas the FR50 would have been compared to the geometric mean for the mixed FR1 FR99 which was 30.

It should be noted, however, that if one translates the procedure used by Stagner and Zentall (2010) into the above comparison of schedules, it would mean that the pigeons preferred a multiple (signaled) FR EXT (extinction) schedule over a mixed FR FR schedule that provided them with 2.5 times as much reinforcement. Furthermore, in the Zentall and Stagner (2011a) study, magnitude of reinforcement was varied rather than probability of reinforcement and according to the marginal value theorem, a fixed magnitude of reinforcement should be preferred over a variable magnitude of reinforcement of the same average magnitude (see e.g., Rachlin, 1992). Moreover, the variable magnitude alternative provided 50% less reinforcement than the fixed magnitude alternative.

However, to ensure that the preference found by Zentall and Stagner (2011a) did not result simply from the pigeons’ preference for variable magnitude of reinforcement (10 pellets 20% of the time) over fixed magnitude of reinforcement (3 pellets 100% of the time), we repeated the experiment and made the discriminative stimuli nondiscriminative. That is, choice of the alternative that provided an average of 2 pellets per trial now produced one of two stimuli, each of which was associated with a 20% chance of providing 10 pellets. The alternative that provided a consistent 3 pellets per trial continued to do so. Under these conditions, the pigeons quickly learned to choose optimally. That is, they now showed a strong preference for the alternative associated with 3 pellets per trial. Thus, it was not the variability of reinforcement associated with the 20% reinforcement alternative that was responsible for the preference for that alternative but the discriminative stimuli that followed that choice.

5. What is the mechanism responsible for suboptimal choice by pigeons?

Why do pigeons prefer discriminative stimuli associated with an overall lower probability of reinforcement over nondiscriminative stimuli associated with a higher probability of reinforcement? Dinsmoor (1983) argued that conditioned reinforcement together with reinforced observing behavior was responsible. Any stimulus that predicts reinforcement with a high probability (in this case 100%) will become a conditioned reinforcer and will elicit observing behavior. Similarly, delay reduction theory (Fantino, 1969) predicts that any stimulus whose presence is associated with a reduction in the delay to reinforcement (relative to its absence) will become a conditioned reinforcer. Although it is clear that such a stimulus should be preferred over a stimulus that predicts reinforcement only 50% of the time (Stagner & Zentall, 2010) or even 75% of the time (Gipson et al., 2009) the question that remains is why the stimulus that was never associated with reinforcement (the S−) showed little evidence of developing conditioned inhibition, especially given that in the Stagner and Zentall study, the S− was presented four times as often as the stimulus that was always followed by reinforcement.

If the S− failed to become a conditioned inhibitor it could have been because, on a given trial, once it was identified as the S− it is possible that the pigeon turned away from it, thus reducing its inhibitory effect (i.e., it maintained little observing behavior; see Dinsmoor, 1985). Roberts (1972) has shown that the effectiveness of a discriminative stimulus is directly related to the duration that it is observed. Consistent with this possibility, the pigeons in Gipson et al., 2009, Stagner and Zentall (2010), and Zentall and Stagner (2011) rarely pecked at the S−, whereas in each of those experiments they pecked at all of the stimuli that were followed by reinforcement. Interestingly, however, Dinsmoor found that when pigeons were presented with an S− and they were able to turn it off (but turning it off did not change the schedule of reinforcement that was in effect), they did so. Thus, the S− stimulus did appear to have some inhibitory properties.

One could test the hypothesis that the S− failed to become an adequate conditioned inhibitor because of a reduction in observing behavior to the S− stimulus by using a diffuse stimulus such as a houselight as the S− stimulus. If the failure to observe or remain in the presence of the S− stimulus is responsible for the preference for the alternative providing less reinforcement, pigeons that are exposed to a diffuse stimulus that signals the absence of reinforcement should develop more inhibition to the S− and thus, should show a preference for the alternative associated with the higher probability of reinforcement. When we conducted such a study, we found that the pigeons continued to prefer the discriminative stimulus alternative associated with an overall lower probability of reinforcement, and did so similar to controls for which the diffuse stimulus served as the S+ (the conditioned reinforcer) as well as for controls for which neither the S+ nor the S− was a diffuse stimulus (Stagner, Laude, & Zentall, 2011).

5.1 Does the S− stimulus fail to produce sufficient inhibition?

More direct procedures to assess the inhibition associated with a stimulus that is followed by the absence of reinforcement have been described by Hearst, Besley, and Farthing (1970). One of these procedures involves the presentation of a compound consisting of a known conditioned reinforcer (S+), together with the presumed conditioned inhibitor. Evidence for conditioned inhibition is found when responding to the S+ decreases when the S− is presented in compound with the S+. To devise such a test with the Stagner and Zentall (2010) design (20% vs. 50% reinforcement) we used a shape (a vertical line) as the S− rather than a color (Laude & Zentall, in press). We then tested the pigeons with both the S+ and the S+/S− compound, first early in training before the appearance of suboptimal choice, and again later in training after the appearance of suboptimal choice. In one experiment we used a repeated measures design and in another a between groups design. Consistent with the hypothesis that with training the S− loses its inhibitory strength, in both experiments, early in training we found that combining the S− with the S+ resulted in a large drop in responding to the S+, whereas later in training there was a significantly smaller drop in responding.

Consistent with these results, a theory based on the absence of conditioned inhibition to losses also has been proposed to account for human gambling behavior. For example, Breen and Zuckerman (1999) reported that humans who gamble regularly have been found to attend more to their wins and less to their considerably more frequent losses than occasional gamblers. Similarly, research with humans has found that problem gamblers show reduced sensitivity to aversive conditioning (Brunborg, Johnsen, Pallesen, Molde, Mentzoni, & Myrseth, 2010) which should also serve to inhibit behavior.

5.2 Reduced response cost associated with the S− stimulus

Another account of the preference for 20% reinforcement over 50% reinforcement by pigeons is that choice of the 50% reinforcement alternative but not the 20% reinforcement alternative results in a considerable amount of nonreinforced responding (Dinsmoor, 1983; Roper & Zentall, 1999). Choice of the 20% reinforcement alternative results in very little nonreinforced pecking because pecking to the S+ is always reinforced, whereas there is generally very little pecking to the S−. On the other hand, on half of the trials involving the 50% reinforcement alternative there is nonreinforced pecking. Although this hypothesis provides a reasonable account of the data from Gipson et al. (2009) and Stagner and Zentall (2010) it has more difficulty accounting for the data from Zentall and Stagner (2011) because reinforcement followed all choices of the alternative associated with the nondiscriminative stimuli. However, those data too could be explained in terms of the cost of pecking per unit of food. If one assumes that pecking is somewhat aversive and that the pigeons peck almost as much at stimuli that predict 3 pellets of food as those that predict 10 pellets of food, the cost per pellet of pecking for 3 pellets of food would be greater than the cost per pellet of pecking for 10 pellets of food.

Although the assumption that pecking is somewhat aversive seems reasonable, in fact, when pecking is required, it is typically confounded with delay of reinforcement. That is, pigeons will prefer pecking less over pecking more if pecking less gets them reinforcement faster. But what if the time to reinforcement is held constant? Delay reduction theory (Fantino & Abarca, 1983) is based on the notion that delay to reinforcement rather than pecking (or effort) determines preference. In support of delay reduction theory, we have found that in the absence of differential delay to reinforcement, pigeons do not necessarily prefer not pecking over pecking (Singer, Berry, & Zentall, 2007). When pigeons were given a choice between pecking (the first response after a specified interval of time was associated with reinforcement, fixed interval) and refraining from pecking (the absence of pecking for a specified interval of time was associated with reinforcement, differential reinforcement of other behavior) and the time to reinforcement was carefully controlled, most pigeons were indifferent between the two schedules, and of the pigeons that did show a preference, it was not always a preference to refrain from pecking. Thus, nonreinforced responding (or responding leading to a lower magnitude of reinforcement) to terminal link stimuli is not likely responsible for the choice of the initial link leading to the lower probability of reinforcement.

5.3 Choice is determined by the value of the S+ stimulus

If the inhibitory value of the conditioned inhibitory stimulus plays a minimal role in choice of the suboptimal alternative, then it must be the value of the conditioned reinforcer, rather than the overall probability of reinforcement associated with choice of each alternative, that is responsible for suboptimal choice. Thus, in the Gipson et al. (2009) study, pigeons do not appear to be choosing between 50% reinforcement associated with the suboptimal alternative and 75% reinforcement associated with the optimal alternative. Instead they appear to be choosing between the conditioned reinforcer associated with 100% reinforcement and the conditioned reinforcer associated with 75% reinforcement. Similarly, in the Stagner and Zentall (2010) study, pigeons do not appear to be choosing between 20% reinforcement associated with the suboptimal alternative and 50% reinforcement associated with the optimal alternative. Instead they appear to be choosing between the conditioned reinforcer associated with 100% reinforcement and the conditioned reinforcer associated with 50% reinforcement. Likewise, in the Zentall and Stagner (2011) study pigeons do not appear to be choosing between an average of 2 pellets associated appear to be choosing between the conditioned reinforcer associated with 10 pellets and the conditioned reinforcer associated with 3 pellets.

This analysis assumes that it is the value of the conditioned reinforcer that follows choice, rather than the frequency of reinforcement associated with that choice, that determines whether the pigeons will choose suboptimally and it leads to in interesting prediction. If pigeons are given a choice between two alternative, one which leads to discriminative stimuli with the S+ occurring 50% of the time (and the S− occurring 50% of the time) and the other which leads to an S+ that occurs 100% of the time, the pigeons should be relatively indifferent between the two alternatives. Interestingly, there is published support for this prediction (Belke & Spetch, 1994; Fantino, Dunn, & Meck, 1979; Mazur, 1996; Spetch, Belke, Barnet, Dunn, & Pierce, 1990; Spetch, Mondloch, Belke, & Dunn, 1994).

Although it is generally found that some pigeons prefer the alternative that provides 50% reinforcement, others prefer the optimal alternative that that provides 100% reinforcement. However, what appear to be individual differences in optimal versus suboptimal choice may actually be produced by an artifact. In the research cited, the fact that the choice was between two alternatives that are defined solely by their spatial location may result in schedule indifference but a spatial preference. We tested this hypothesis and the hypothesis that conditioned reinforcers with comparable value would produce indifference, independent of the probability of their occurrence (Stagner, Laude, & Zentall, 2012). Pigeons were given a choice between two alternatives, one which led to discriminative stimuli with the S+ occurring 20% of the time (and the S− occurring 80% of the time) and the other which led to discriminative stimuli with the S+ occurring 50% of the time (and the S− occurring 50% of the time). To avoid spurious schedule preferences that might be attributed to spatial preferences, to signal the two alternatives in the initial link, we used line orientation stimuli (vertical or horizontal lines), the location of which (left or right) varied randomly on choice trials, (see Figure 5). The results confirmed our prediction. All of the pigeons were virtually indifferent between the two alternatives. To confirm that the pigeons could discriminate between the two schedules, when the stimuli that followed the alternative associated with 50% reinforcement were made nondiscriminative, a strong suboptimal choice effect was found, whereas when the stimuli that followed both alternatives were made nondiscriminative, a strong optimal choice effect was found. Thus, the value of conditioned reinforcers play an important role in the suboptimal choice found for pigeons.

Figure 5.

Pigeons chose between two alternatives that were distinguished by discriminative stimuli (a vertical or a horizontal line). Choice of one alternative was followed either by a stimulus (e.g., red) on 20% of the trials that was always followed by reinforcement or by a different stimulus (e.g., green) on 80% of the trials that was never followed by reinforcement. Choice of the other alternative was followed either by a stimulus (e.g., blue) on 50% of the trials that was always followed by reinforcement or by a different stimulus (e.g., yellow) on 50% of the trials that was never followed by reinforcement. Spatial location and colors were counterbalanced.

Furthermore, conditioned reinforcers also appear to play an important role in human suboptimal choice by humans. This conclusion is supported by a line of research on observing behavior which shows that humans will work to obtain a signal for reinforcement but not a signal for the absence of reinforcement when neither changes the probability of reinforcement. For example, Fantino and Case (1983; see also Fantino & Silberberg, 2010) exposed subjects to a mixed variable time (response independent), extinction schedule in which in one condition, responses produced a stimulus which signaled that the variable time schedule was in effect (a presumed conditioned reinforcer) and a stimulus which signaled that the extinction schedule was in effect (a presumed conditioned inhibitor). In a second condition, responses produced only the presumed conditioned reinforcer (when it was in effect) and in a third condition, responses produced only the presumed conditioned inhibitor (when it was in effect). Importantly, in no case did responding have any effect of the schedule itself; it only identified the schedule that was already in effect. Fantino and Case found that subjects would respond to produce the stimulus associated with reinforcement but not to produce the signal for nonreinforcement, even though they both contain the same amount of information (i.e., a conditioned reinforcer lets the subject know that reinforcement can be obtained whereas a conditioned inhibitor lets the subject know that reinforcement cannot be obtained).

The results of these experiments with pigeons and humans are consistent with the findings from human gambling research that conditioned reinforcers play an important role for problem gamblers (Crockford, Goodyear, Edwards, Quickfall, el-Guebaly, 2005), whereas conditioned inhibitors exert very little control over the their decisions to gamble (Field and Cox 2008; Franken, Stam, Hendriks, and van den Brink, 2003; Holst, van den Brink, Veltman, and Goudriaan 2010; Tversky and Kahneman 1974). The results of these experiments suggest that one approach to the treatment of pathological gamblers might be to make them more aware of their losses my making their losses more salient.

6. Is the pigeon task a reasonable model of human gambling behavior?

6.1 What would humans do?

The task we have developed for pigeons is proposed to be an animal model of human gambling because pigeons like gamblers show suboptimal behavior. That this task is analogous to human gambling would be further supported if one could show that the performance by humans on a similar task was correlated with the degree to which those individuals engaged in gambling behavior. Using a modified version of the pigeon task used by Zentall and Stagner (2011a), Molet, Miller, Laude, Kirk, Manning, and Zentall (2011) tested this prediction and found that humans who self-reported that they regularly engaged in commercial gambling, chose the suboptimal alternative significantly more than nongamblers. Thus, this suboptimal choice task can be thought of as diagnostic of gambling behavior and it suggests that variables found to affect pigeons’ choice of the suboptimal alternative may have implications for humans who gamble. Furthermore, the results of Molet et al. suggest that mechanisms found to be involved in suboptimal choice by pigeons may also be relevant to human gambling.

6.2 Task differences from human gambling

6.2.1 Go/no-go versus two-alternative forced choice

When humans gamble it is the equivalent of a go/no-go task because humans can choose to gamble with money that they already have or refrain from gambling. Pigeons, on the other hand, choose between an optimal outcome and a suboptimal outcome, neither of which they already have. This distinction should make it even more likely that humans would not gamble because for humans, not only do they have a choice between a sure outcome and a probabilistic outcome but the sure outcome is immediate (money already in their pocket) whereas the probabilistic outcome is delayed by the time it takes to gamble and learn about the outcome. This may explain why only a small percentage of humans are actually problem gamblers. In fact, we have found that for pigeons, if the suboptimal outcome is delayed, relative to the optimal outcome, the pigeons begin to choose optimally (Zentall & Stagner, 2011b, Exp. 1).

Because humans choose to gamble with money that they already have, unlike pigeons their losses represent actual losses rather than the absence of reinforcement. This distinction may be important because according to prospect theory (Kahneman & Twersky, 1979), although gains that are certain (e.g., the certainty of winning $5) are preferred over proportionally larger gains that are probabilistic (e.g., an 80% chance of winning $10)(the certainty effect), losses that are certain (e.g., the certainty of losing $5) are avoided over proportionally larger losses that are probabilistic (e.g., an 80% chance of losing $10)(the reflection effect). That is, there is a stronger bias to win back losses than to obtain gains, an effect that typically encourages gamblers to keep gambling.

Although it would be difficult to create a task in which pigeons, like humans, can choose to gamble with a reinforcer that they already have, as already noted, we have tested humans who are self-reported gamblers on a version of the pigeon two-alternative choice task and found that they are more likely to choose suboptimally than self-reported non gamblers (Molet et al., 2011). Thus, the difference between the go/no-go choice provided by commercial gambling and the two-alternative choice provided by our analog task does not appear to be responsible for the suboptimal choice by pigeons.

6.2.2 The role of conditioned reinforcers in human gambling

One of the features of the suboptimal choice task used with pigeons is the appearance of conditioned reinforcers following choice but prior to the appearance of the outcome. Are there analogous conditioned reinforcers present in human gambling? Although it is not obvious that conditioned reinforcers are present, the results of a thought experiment suggests that they are. For example, the three reels on a slot machine can be thought of as conditioned reinforcers. Evidence that they serve that role is suggested by the likelihood that people would not tend to gamble nearly as much if the reels on the slot machine could not be seen. That is, if the only outcome of money inserted in the machine would be either nothing or money falling into the coin tray, is there any doubt that people would be less likely to play? A similar argument can be made for other games of chance (e.g., roulette and black jack). Thus, although there may be some procedural differences between the pigeon suboptimal choice task and human commercial gambling, the important elements of the two are quite similar as supported by the finding that a modified version of the pigeon task is able to distinguish human gambler from nongamblers (Molet et al., 2011).

7. The demographics of gambling behavior

7.1 The relation between level of food restriction and suboptimal choice

A paradoxical demographic of human gambling behavior is that people with higher needs (those of lower socio-economic status) tend to gamble proportionally more than those with lower needs (those of higher status) (Lyk-Jensen, 2009; Worthington, 2001). If our pigeon model of suboptimal choice is a reasonably good analog of human gambling behavior, the level of pigeons’ food motivation should predict their degree of suboptimal choice. Laude, Pattison, and Zentall (2012) tested this hypothesis and found support for the relationship. They found that pigeons that were minimally food restricted chose optimally, whereas those that were normally food restricted showed the typical suboptimal choice.

The mechanism responsible for this suboptimal choice effect is likely to be impulsivity. Impulsivity has been proposed to be associated with human suboptimal choice involves in gambling (Michalczuk, Bowden-Jones, Verdejo-Garcia, & Clark, 2011; Nower, & Blaszczynski, 2006). Impulsivity has been defined as the inability to delay reinforcement and it has been assessed by way of delay discounting tasks in which an organism is given a choice between a small immediate reinforcement and a larger delayed reinforcement. The delay at which the organism is indifferent between the two alternatives defines the slope of the discounting function and the degree of impulsivity. Thus, impulsive individuals require that the delay to the larger amount of reinforcement be relatively short before they will prefer it and thus for them the slope of the discounting function would be relatively steep. We have recently found that the slope of the delay discounting function for pigeons is a good predictor of the degree to which they prefer the suboptimal choice in the gambling-like task (Laude, Beckmann, Daniels, & Zentall, 2013).

7.2 The relation between housing and suboptimal choice

There is some suggestion from research with rats that various extra-experimental environmental factors such as social and physical enrichment can affect a rat's propensity to self-administer drugs of addiction (Stairs and Bardo 2009). Rats that are housed in an enriched group environment (a large cage with other rats and objects that are changed regularly) show a significantly reduced tendency to self-administer drugs than rats that are normally (individually) housed. The mechanism responsible for the reduced self-administration of drugs by environmental enrichment appears to be a reduction in impulsive behavior (Perry and Carroll 2008) as well as the reduced effectiveness of conditioned reinforcers (Jones, Marsden, and Robbins 1990). Impulsivity has also been implicated in human gambling behavior (Steel and Blaszczynski, 1998) and, as already noted, conditioned reinforcement has been proposed to account for suboptimal choice by animals (Dinsmoor, 1983). Furthermore, there is evidence that similar physiological mechanisms underlie compulsive gambling and drug addiction (Potenza, 2008).

In an attempt to determine the effect of housing on suboptimal choice, we gave one group of pigeons experience in an enriched environment (a large cage with four other pigeons for 4 hr a day), while the control pigeons remained in their normal one-to-a-cage housing. When we exposed the pigeons from both groups to the gambling-like task we found that the normally housed pigeons showed the typical suboptimal choice, whereas the enriched pigeons initially chose optimally (for about 10 sessions) but with further training they began to choose suboptimally as well. Thus, enriched housing appears to have an effect on suboptimal choice, even if that effect may be only temporary. The implications for the treatment of problem gambling behavior by humans are clear. If this finding can be generalized, it implied that exposing human gamblers to an environment that is socially and physically enriched may reduce the attraction of gambling.

8. Conclusions

To explain why pigeons prefer discriminative stimuli over nondiscriminative stimuli, Dinsmoor (1983) proposed that animals are attracted to conditioned reinforcers, but we have argued that the stimuli associated with 0% reinforcement should result in conditioned inhibition and in the case of 20% reinforcement those nonreinforcements occur four times as often. We now know that although Dinsmoor ignored conditioned inhibition he was probably correct to do so because the conditioned inhibitors are relatively ineffective, even when they occur four times as often as the conditioned reinforcer.

Furthermore, we now know that the probability of the occurrence of the conditioned reinforcer is relatively unimportant as well (Stagner et al., 2012). That is, the probability of winning is relatively unimportant. This finding has implications for human gambling behavior. If the probability of the appearance of the conditioned reinforcer is relatively unimportant, it provides a plausible reason for why humans gamble when the odds of losing are very high (lotteries). Those who run casinos and lotteries have found a way to get people to gamble, even though they have never won, by drawing attention to winning by others (bells ringing and lights flashing when there is a slot-machine winner in a casino and an announcement on TV when there is a lottery jackpot winner). By doing this, they make it appear that winning is much more likely than it is (the availability heuristic).

Why the unimportance of losing exists in humans and other animals is not clear but it may have had an evolutionary adaptive value. If food is scarce, there may be many more failures than successes to find food. But developing inhibition to searching generally would not be adaptive. Thus, in nature, it may be more adaptive to disregard or at least deemphasize losses.

A second reason that animals may be attracted to low probability but high valued outcomes is that their attraction, which generally takes the form of approach behavior is likely to have an effect on later outcomes. The edges of a foraging patch may have high valued outcomes with low probability but entering the patch may result in an increase in the probability of those outcomes.

Gambling may be a different activity in part because choice of the low probability but high valued outcome does not change the probability of obtaining it as it might in nature but it appears to elicit a similar response. The present research suggests that one approach to the treatment of problem gambling may be to make wins less salient and, perhaps more important and easier to accomplish, make losses more salient.

The present research also suggests that changes in environment conditions may affect gambling. It may be difficult to overcome the greater tendency to gamble by those humans with lower socio-economic status because although the real cost of gambling for those people is relatively higher than for those with higher socio-economic status, the possibility of winning a jackpot, would presumably represent a greater improvement in life style for those who are poor. On the other hand, it may be possible to affect gambling by making other changes in the environment. It is not clear whether problem gamblers spend as much time as they do gambling because they have few outside interests or that problem gambling results in having few outside interests, however, the finding with pigeons that environmental enrichment can reduce the attraction to the suboptimal alternative suggests the possibility that exposing humans to other enriching activities may also serve to reduce their attraction to gambling.

9. Further research

The fact that humans who gamble also show more suboptimal choice on a version of the pigeon task than humans who do not gamble lends support to the assumption that the suboptimal choice task provides a good analog to human gambling behavior. The pigeon model of suboptimal choice may be useful to study several variables that may be important in humans gambling behavior. For example, it would be of interest to examine the effects of stimulus salience (both the conditioned reinforcer and the conditioned inhibitor) on suboptimal choice. Also, it is possible that the magnitude of reinforcement (independent of probability of reinforcement,) may be found to affect suboptimal choice because there is evidence that the larger the jackpot the more likely humans are to gamble.

The suboptimal choice tasks can also be used to study the curious effect of the “near win” (losses that come close to winning – losing a lottery by one number – appear to encourage further gambling) in spite of the fact that one might think that the similarity between the loss associated with the near hit and a win might serve to devalue a win. It is possible that this near win effect is unique to humans and may be attributable to a false sense of control that some humans have in commercial games of chance.

Highlights.

• A pigeon model of human gambling behavior

• As with human gambling losses play less of a role than they should

• As with human gambling low frequency of wins plays less of a role than it should