Effects of Pre-Trial Response Requirements on Self-Control Choices by Rats and Pigeons

Abstract

Parallel experiments with rats and pigeons examined whether the size of a pre-trial ratio requirement would affect choices in a self-control situation. In different conditions, either 1 response or 40 responses were required before each trial. In the first half of each experiment, an adjusting-ratio schedule was used, in which subjects could choose a fixed-ratio schedule leading to a small reinforcer, or an adjusting-ratio schedule leading to a larger reinforcer. The size of the adjusting ratio requirement was increased and decreased over trials based on the subject's responses, in order to estimate an indifference point—a ratio at which the two alternatives were chosen about equally often. The second half of each experiment used an adjusting-delay procedure—fixed and adjusting delays to the small and large reinforcers were used instead of ratio requirements. In some conditions, particularly with the reinforcer delays, the rats had consistently longer adjusting delays with the larger pre-trial ratios, reflecting a greater tendency to choose the larger, delayed reinforcer when more responding was required to reach the choice point. No consistent effects of the pre-trial ratio were found for the pigeons in any of the conditions. These results may indicate that rats are more sensitive to the long-term reinforcement rates of the two alternatives, or they may result from a shallower temporal discounting rate for rats than for pigeons, a difference that has been observed in previous studies.

Theoretical analyses of animal choice behavior range from molar approaches, which examine the long-term relationships between behavior and its consequences, to molecular approaches, which assume that behavior is more strongly influenced by immediate consequences than by delayed consequences. Examples of the molar approach include different variations of optimal foraging theory (e.g., Charnov, 1976; Lea, 1979) or optimal diet theory (e.g., Sih & Christensen, 2001), which propose that animals' choices will tend to maximize the overall rate of reinforcement, or to minimize the amount of work or time expended per reinforcer. Among the various molecular theories are those related to delay discounting (e.g., Mazur, 1987; Sopher & Sheth, 2006; van der Pol & Cairns, 2002), which propose that the reinforcing strength or value of a reinforcer decreases as the time between a choice response and reinforcer delivery increases. For some choice situations, the predictions of molar and molecular approaches may be similar. For example, both molar theories (e.g, McDowell, Caron, Kulubekova, & Berg, 2008; Rachlin, Green, Kagel, & Battalio, 1976) and molecular theories (e.g., Brown & Cleaveland, 2009; Hinson & Staddon, 1983; Silberberg, Hamilton, Ziriax, & Casey, 1978) can predict matching (or generalized matching, cf. Baum, 1979) in concurrent variable-interval (VI) schedules. However, for other choice situations, molar and molecular theories make distinctly different predictions.

One area where the predictions of molar and molecular theories differ dramatically is intertemporal choice, in which the individual must choose between reinforcers that are delivered at different times. For example, in self-control choice situations, the individual must choose between a smaller, more immediate reinforcer and a larger, delayed reinforcer. Numerous experiments have shown that both humans (e.g., Green, Myerson, & McFadden, 1997; Odum, Madden, & Bickel, 2002) and nonhumans (e.g., Mazur & Biondi, 2009; Richards, Mitchell, de Wit, & Seiden, 1997; Woolverton, Myerson, & Green, 2007) will often choose the smaller, more immediate reinforcer. These findings clearly seem to favor molecular theories over molar theories, since the way to optimize the amount of reinforcement in the long run would be always to choose the larger, more delayed reinforcer.

Despite the abundant evidence that self-control choices are determined by molecular factors (i.e., the delays to the smaller and larger reinforcers), it is possible that under certain circumstances, these choices might be at least partly controlled by molar contingencies. One possible factor is the amount of work required to reach the point where one must choose between the smaller, sooner and larger, later reinforcers. For example, suppose that on each trial of a choice procedure a rat must make one lever press to start a trial, and then it can choose between a fixed-ratio (FR) 10 schedule that delivers one food pellet (the standard alternative) and an adjusting-ratio schedule that delivers two food pellets (the adjusting alternative). The adjusting ratio is increased or decreased over trials depending on the rat's choices, so as to estimate an indifference point—a ratio at which the two alternatives are chosen equally often. After each trial there is a 20-s intertrial interval (ITI), and then one lever press is again required to reach the next choice point. Suppose an indifference point of 21 responses is obtained. Notice that such a result could be interpreted as supporting either a molar approach (because the long-term response∶reinforcer ratio is 11 for both alternatives) or a molecular approach (the two-pellet alternative is discounted due to the longer time needed to make 22 responses rather than 11 responses).

Now suppose that one change is made in the procedure: After each ITI, 40 lever presses are required to reach the choice point rather than just 1 lever press. What effect, if any, will this have on the rat's indifference point? According to a molar perspective, we might expect the adjusting ratio to increase, for the following reason. If the adjusting ratio remained at 21 responses, this would mean that the rat would make a total of 61 responses to obtain two food pellets (40 responses to reach the choice point, plus 21 after the choice is made), versus 50 responses to obtain one food pellet (40 responses to reach the choice point, plus 10 after the choice is made). The rat should show a preference for the adjusting-ratio alternative because of its lower response/food ratio, and this preference would drive up the adjusting ratio to a higher indifference point. If the adjusting ratio reached 60 responses at the indifference point, this would equalize the response∶reinforcer ratio at 50 total responses per food pellet for both alternatives. A molar theory need not predict this specific quantitative result, because a variety of other factors (e.g., the time needed to switch between levers, the 20-s ITI) could also affect the indifference point. However, according to this molar approach, there should be some increase in the indifference point when the pre-choice response requirement is increased from FR 1 to FR 40.

What would a molecular approach predict for this situation? For many choice situations involving delayed reinforcers and discrete-trial choices, I have found that a molecular theory called the hyperbolic-decay model can make good predictions for animals' choices (e.g., Mazur, 1984, 1987, 1997, 2007). In its most basic form, the model can be written as follows:

where V is the value or strength of a reinforcer, D is the delay between a choice response and the reinforcer, A reflects the amount or size of the reinforcer, and K is a parameter that determines how rapidly V decreases with increases in D. Grossbard and Mazur (1986) showed that this same equation could make accurate predictions for self-control choice situations in which ratio schedules were used rather than delays (similar to the hypothetical example described here). According to this model, the value of each alternative is determined by the delay to reinforcement that remains at the moment the choice response is made, and it is not influenced by any delays that precede the choice response. Therefore, in its simplest form, this model predicts that the pre-trial response requirement should have no effect on the rat's indifference point.

There is, however, an exception to this prediction if the pre-trial response requirement differentially affects the time to food on later trials, and in the present experiments it did. The ITI and the pre-trial ratio schedule were the same after either a standard or an adjusting trial. Therefore, if an animal chose the alternative with the shorter delay to food on one trial, the delay to food would be shorter on all subsequent trials. This is an added benefit of choosing the smaller, sooner reinforcer that is not captured by Equation 1, but it can have an effect on the indifference point. For example, Mazur, Snyderman, and Coe (1985) used an adjusting-delay procedure with pigeons in which there was an ITI after the standard alternative but no ITI after the adjusting alternative. They varied the duration of the ITI across conditions, and found small but consistent increases in the adjusting delays as the ITI after the standard trials was lengthened. This showed that the food reinforcers on subsequent trials had an effect on the pigeons' choices on the present trial. To account for the effects of reinforcers on multiple trials, Mazur et al. used an extension of Equation 1:

The notation is the same as in Equation 1 except that V, the value of an alternative, now includes the reinforcers delivered on n trials, and D_i is the delay from the choice response on the current trial to the reinforcers delivered on subsequent trials. Mazur et al. found that when Equation 2 included the values of the reinforcers delivered on the next several trials, the equation accounted for the effects of variations in ITI duration fairly well.

The findings of Mazur et al. (1985) are relevant to the present experiments in the following way. The delays to reinforcers delivered on subsequent trials will, of course, be greater when there is a large pre-trial response requirement. Because reinforcer values decrease with delay, reinforcers from later trials will contribute less to the value of choosing the smaller, sooner reinforcer when there is a large pre-trial response requirement. The advantage of choosing the smaller, sooner reinforcer should be smaller when reinforcers from subsequent trials are more distant, so Equation 2 predicts that preference for this alternative should decrease. In the adjusting-delay procedure, such a decrease would appear as an increase in the delay for the adjusting alternative (which delivers the larger reinforcer).

The purpose of the present experiments was to determine whether the amount of work that is required to reach a choice point would affect the choice behavior of two different species, rats and pigeons. Similar experiments were conducted with these two species using a series of self-control choice situations, in which the animals chose between a smaller, more immediate amount of food and a larger, more delayed amount of food. In the first half of each experiment, an adjusting-ratio procedure was used similar to the one described above, and the pre-trial response requirement was FR 1 in some conditions and FR 40 in other conditions. These two pre-trial FR schedules were also used in the second half of each experiment, but an adjusting-delay procedure was used instead of an adjusting-ratio procedure: The animals chose between a smaller amount of food after a 2-s delay and a larger amount of food after an adjusting delay. I know of only one previous experiment that has examined the effects of a pre-trial response requirement on self-control choice. In three conditions of a larger experiment using an adjusting-delay procedure to study self-control choice, Mazur (1988) used pre-trial response requirements of FR 1, FR 10, and FR 30, and there was no systematic effect of this manipulation on the pigeons' indifference points. The present set of experiments was designed to examine this matter more thoroughly, and to compare the performances of pigeons and rats. Results from a number of previous experiments have suggested that the rates of delay discounting are faster for pigeons than for rats by about a factor of four or five (e.g., Green, Myerson, Holt, Slevin, & Estle, 2004; Mazur, 2000, 2007; Mazur & Biondi, 2009; Richards et al, 1997). It is therefore possible that rats might be more affected than pigeons to the long-term detrimental effects of a pre-trial response requirement on the rate of food delivery across a series of trials.

These experiments should provide information about the relative influences of molecular and molar variables in controlling choice. If the pre-trial ratio has no effect whatsoever on the subjects' indifference points, this would suggest that their choice behavior is solely controlled by molecular variables (i.e., the delay to the next reinforcer). However, if indifference points increase with the larger pre-trial ratio schedule, this could reflect the effects of molecular variables (i.e., the response/reinforcer ratios for the two alternatives), particularly if these effects are too large to be explained by the reinforcers delivered on subsequent trials (cf., Mazur et al., 1985).

EXPERIMENT 1

Method

Subjects

This experiment used 4 male Long Evans rats approximately 5 months old at the start of the experiment. The rats were maintained at 80% of their free-feeding weights. They had previously received experience responding on all three levers in pilot research, so no additional pretraining was needed.

Apparatus

The experimental chamber was a modular test chamber for rats, 30.5 cm long, 24 cm wide, and 21 cm high. The side walls and top of the chamber were Plexiglas, and the front and back walls were aluminum. The floor consisted of steel rods, 0.48 cm in diameter and 1.6 cm apart, center to center. The front wall had two retractable response levers, 11 cm apart, 6 cm above the floor, 4.8 cm long, and extending 1.9 cm into the chamber. Centered in the front wall was a nonretractable lever with the same dimensions, 11.5 cm above the floor. A force of approximately 0.20 N was required to operate each lever, and when a lever was active, each effective response produced a feedback click. Above each lever was a 2-W white stimulus light, 2.5 cm in diameter. A pellet dispenser delivered 45-mg food pellets into a receptacle through a square 5.1 cm opening in the center of the front wall. A 2-W white houselight was mounted at the top center of the rear wall.

The chamber was enclosed in a sound-attenuating box containing a ventilation fan. All stimuli were controlled and responses recorded by an IBM-compatible personal computer using the Medstate programming language.

Procedure

The experiment consisted of four phases of four conditions each. An adjusting-ratio procedure was used in Phases I and II, and an adjusting-delay procedure was used in Phases III and IV.

Phase I (Conditions 1–4)

In this phase and throughout the experiment, each session lasted 64 trials or 60 min, whichever came first. Each block of four trials consisted of two forced trials followed by two choice trials. Before each trial there was a 20-s ITI during which the white houselight was lit. At the end of the ITI, the houselight was turned off, the light above the center lever was lit, and an FR response requirement was in effect on the center key. The schedule was FR 1 in Conditions 1 and 3, and it was FR 40 in Conditions 2 and 4.

On choice trials, after the required number of responses on the center lever, the light above this lever was turned off, the two side levers were extended into the chamber, and the lights above the two side levers were turned on. A single response on the left lever constituted a choice of the standard alternative, and a single response on the right lever constituted a choice of the adjusting alternative. If the standard (left) lever was pressed during the choice period, the right lever was retracted and the light above it was turned off. The left lever remained in the chamber and the light above it remained on until the rat completed an FR 10 response requirement. Then, the left lever was retracted and the light above it was turned off, one food pellet was delivered, and the chamber was dark for 1 s. The white houselight was then lit and a 20-s ITI began.

If the adjusting (right) lever was pressed during the choice period, the left lever was retracted and the light above it was turned off. The right lever remained in the chamber and the light above it remained on until the rat completed an adjusting-ratio response requirement, as explained below. Then, the right lever was retracted and the light above it was turned off, two food pellets were delivered, and the chamber was dark for 1 s. The white houselight was then lit and a 20-s ITI began.

The procedure on forced trials was the same as on choice trials, except that only one side lever was extended and the light above it was lit. One press on this lever was followed by the same sequence of events as on a choice trial. Of every two forced trials, there was one for the standard lever and one for the adjusting lever, and the temporal order of the two types of trials varied randomly.

After every two choice trials, the size of the adjusting ratio might be changed. If the rat chose the standard lever on both choice trials, the adjusting ratio was decreased by two responses. If the rat chose the adjusting lever on both choice trials, the adjusting ratio was increased by two responses. If the rat chose each lever on one trial, no change was made. In all three cases, this adjusting ratio remained in effect for the next block of four trials. At the beginning of the first session of a condition, the adjusting ratio was set at two responses. At the beginning of later sessions of the same condition, the adjusting ratio was determined by the above rules as if it were a continuation of the preceding session.

Condition 1 lasted for a minimum of 20 sessions, and all other conditions lasted for a minimum of 12 sessions. However, between Conditions 1 and 2, the pre-trial response requirement on the center key was gradually increased from FR 1 to FR 40 over approximately 32 sessions. The data from these transition sessions were not used in any analyses. After the minimum number of sessions, a condition was terminated for each rat individually when several stability criteria were met. To assess stability, each session was divided into two 32-trial blocks, and for each block the mean adjusting ratio was calculated. The results from the first two sessions of a condition were not used, and the condition was terminated when the following criteria were met, using the data from all subsequent sessions: (a) Neither the highest nor the lowest single-block mean of a condition could occur in the last six blocks of a condition. (b) The mean adjusting ratio across the last six blocks could not be the highest or the lowest six-block mean of the condition. (c) The mean adjusting ratio of the last six blocks could not differ from the mean of the preceding six blocks by more than 10% or by more than one response (whichever was larger). Table 1 shows the number of sessions needed by each rat to meet the stability criteria in each condition.

Table 1.

Order of conditions and number of sessions needed to meet the stability criteria for each rat in Experiment 1. ITI  =  intertrial interval; FR  =  fixed ratio; FT  =  fixed time.

Phase II (Conditions 5–8)

The procedure in this phase was the same as in Phase I, except that the standard ratio requirement was reduced from FR 10 to FR 1. That is, one response on the standard lever was required to choose that alternative, and a second response led to the delivery of one pellet. The pre-trial response requirement on the center key was FR 1 in Conditions 5 and 7, and it was FR 40 in Conditions 6 and 8. The purpose of this phase was to determine whether the effects of the pre-trial response requirement would be greater if it constituted a greater proportion of the total responses that were needed to obtain food on each trial.

Phase III (Conditions 9–12)

The procedure in this phase was the same as in Phase I, except that the standard and adjusting ratio requirements were replaced with standard and adjusting delays. The stimuli and response requirements were the same as in Phase I until a choice response was made. Then, if the standard (left) lever was pressed, both levers were retracted, and only the light above the left lever remained on for a 2-s delay. At the end of the delay, one food pellet was presented and the chamber was dark for 1 s. If the adjusting (right) lever was pressed during the choice period, both levers were retracted, and only the light above the right lever remained on for an adjusting delay. At the end of the adjusting delay, two food pellets were presented and the chamber was dark for 1 s. A 20-s ITI with the white houselights on followed the pellet deliveries on both standard and adjusting trials.

The rules for changing the adjusting delay were the same as those for changing the adjusting ratio in the first two phases, except that the adjusting delay was increased or decreased in 1-s increments. The criteria for ending each condition were also similar. Each condition continued for a minimum of 12 sessions. Each session was divided into two 32-trial blocks, and for each block the mean adjusting delay was calculated. The results from the first two sessions of a condition were not used, and the condition was terminated when the following criteria were met, using the data from all subsequent sessions: (a) Neither the highest nor the lowest single-block mean of a condition could occur in the last six blocks of a condition. (b) The mean adjusting delay across the last six blocks could not be the highest or the lowest six-block mean of the condition. (c) The mean adjusting delay of the last six blocks could not differ from the mean of the preceding six blocks by more than 10% or by more than 1 s (whichever was larger).

Phase IV (Conditions 13–16)

The procedure in this phase was the same as in Phase III, except that the ITI was decreased from 20 s to 3 s. The reasoning behind this manipulation was that with a shorter ITI, the pacing of the trials would largely depend on whether the pre-trial response requirement was FR 1 or FR 40. Therefore, the effects of varying the pre-trial response requirement might be more apparent in this phase. The response requirement was FR 1 in Conditions 13 and 15, and it was FR 40 in Conditions 14 and 16.

Results and Discussion

Table 1 shows the number of sessions needed by each rat to reach the stability criteria in each condition. All data analyses were based on the results from the six half-session blocks that satisfied the stability criteria. For each rat and each condition, the mean adjusting ratio (in Phases I and II) or mean adjusting delay (in Phases III and IV) from these six half-session blocks was used as a measure of the indifference point.

For each of the 4 rats, Figure 1 presents the indifference points (mean adjusting ratios) from the eight conditions in Phases I and II. As expected, the mean adjusting ratios were larger in most cases when the standard schedule was FR 10 (Phase I, M  =  62.7 responses) than when the standard schedule was FR 1 (Phase II, M  =  43.8 responses). The white bars are the results from conditions with the FR 1 pre-trial response requirement, and the black bars are from conditions with the FR 40 pre-trial response requirement. The effects of the pre-trial response requirement varied greatly across subjects. When the standard schedule was FR 10, only Rat R1 showed a clear effect in the direction predicted by a molar approach: Averaged across replications, the mean adjusting ratio for this rat was 33.3 when the pre-trial schedule was FR 1 and 103.7 when the pre-trial schedule was FR 40. The other 3 rats showed no systematic effects of the pre-trial schedule. However, when the standard schedule was reduced to FR 1 in Phase II, Figure 1 shows a systematic effect of the pre-trial schedule for 3 of the 4 rats (with Rat R2 as the exception). For these 3 rats, the mean adjusting ratio was consistently larger when the pre-trial schedule was FR 40 than when it was FR 1.

Fig 1.

The mean adjusting ratios are shown for each rat in Experiment 1, from Phase I (where the standard schedule was FR 10) and Phase II (where the standard schedule was FR 1).

Response rates on the ratio schedules were examined to determine whether the differences among the rats might be related to differences in how long it took the rats to complete the various ratio requirements. In Phase I, Rat R1 was the only subject to show an effect of the pre-trial FR requirement, but this rat's response rates on the pre-trial FR 40, the standard FR 10, and the adjusting-ratio schedule were all within the range of response rates found for the other 3 rats. In Phase II, Rat R2 was the only subject that did not show a systematic effect of the pre-trial FR requirement. Rat R2 did have the highest response rate on the adjusting-ratio schedule (4.4 responses/s, compared to 2.5, 1.9, and 3.3 responses/s for the other 3 rats). However, it is not clear why a faster response rate on the adjusting-ratio schedule would make the rat less sensitive to the differences in the pre-trial FR schedule. This rat's response rate on the pre-trial FR 40 schedule was similar to those of the other 3 rats. In short, there was little evidence that individual differences in response rates were responsible for the different results shown in Figure 1.

When delays were used as the standard and adjusting alternatives in Phases III and IV, the effects of the pre-trial response requirement were clearer. Figure 2 shows the mean adjusting delays from these eight conditions. In Phase III, the mean adjusting delay for the group was 12.0 s in conditions with the pre-trial FR 1, and it was 19.6 s in conditions with the pre-trial FR 40. In Phase IV, the group average was 4.6 s in conditions with the pre-trial FR 1, and it was 15.5 s in conditions with the pre-trial FR 40. As can be seen in Figure 2, the mean adjusting delays varied greatly across subjects. However, for all rats in Phases III and IV, the adjusting delays were, without exception, shorter in each of the two conditions with the pre-trial FR 1 than in the two conditions with the pre-trial FR 40.

Fig 2.

The mean adjusting delays are shown for each rat in Experiment 1, from Phase III (where the ITI was 20 s) and Phase IV (where the ITI was 3 s).

In summary, only 1 rat showed consistently longer indifference points with the pre-trial FR 40 in Phase I, but 3 rats showed this effect in Phase II, and all 4 rats showed the effect in Phases III and IV. As noted in the Method section, the conditions were more favorable to finding such an effect in Phase II than in Phase I because of the shorter FR schedule for the standard alternative in Phase II. Similarly, the conditions were more favorable to finding such an effect in Phase IV than in Phase III because of the shorter ITI in Phase IV. However, it is not clear why the effect of the pre-trial ratio was found more reliably when the choices were two different delays to food (Phases III and IV) rather than two ratio schedules (Phases I and II). One possibility is that the difference between the pre-trial FR 1 and FR 40 was more salient in the phases with delays because there were no other ratio requirements during these sessions, whereas in the first two phases, the rats had to complete ratio requirements both before and after the choice response. There is also the possibility of an order effect, because the two phases with delays occurred after the two phases with the ratio schedules.

Either way, this experiment showed that, at least under some conditions, rats' choices in a self-control procedure are sensitive to the size of a pre-trial response requirement. Experiment 2 examined whether pigeons would show a similar effect.

EXPERIMENT 2

Method

Subjects

The subjects were 8 male white Carneau pigeons maintained at about 80% of their free-feeding weights. All pigeons had previous experience with a variety of experimental procedures.

Apparatus

Two identical experimental chambers were used. Each chamber was 30 cm long, 30 cm wide and 31 cm high. The chambers had three response keys, each 2 cm in diameter, mounted on the front wall of the chamber, 24 cm above the floor and 8 cm apart. A force of approximately 0.15 N was required to operate each key. Each key could be transilluminated with lights of different colors. A hopper below the center key delivered controlled access to grain (whole-grain wheat), and when the grain was available, the hopper was illuminated with a 2-W white light. Eight 2-W houselights (two white, two green, two blue, and two red) were mounted in a row above the Plexiglas ceiling toward the rear of the chamber. Each chamber was enclosed in a sound-attenuating box with a ventilation fan. All stimuli were controlled and responses were recorded using an IBM compatible computer using the Medstate programming language.

Procedure

This experiment was designed to be very similar to Experiment 1, except that the pigeons responded on three response keys rather than on levers, and the small and large reinforcers were 2 s and 6 s of access to grain rather than food pellets. Each session lasted 64 trials or 60 min, whichever came first. Each block of four trials consisted of two forced trials followed by two choice trials. As shown in Table 2, the 16 conditions of this experiment included the same pre-ratio requirements, standard and adjusting delays or ratios, and ITI durations as in the corresponding conditions in Experiment 1. Condition 1 lasted for a minimum of 25 sessions, and all other conditions lasted for a minimum of 12 sessions. The criteria for terminating each condition were the same as in Experiment 1.

Table 2.

Order of conditions and number of sessions needed to meet the stability criteria for each pigeon in Experiment 2. ITI  =  intertrial interval; FR  =  fixed ratio; FT  =  fixed time.

Phases I and II (Conditions 1–8)

Before each trial there was a 20-s ITI during which the white houselights were lit. At the end of the ITI, the houselights were turned off, and the center key was transilluminated with white light, and an FR response requirement was in effect on the center key. The pre-trial response requirement was FR 1 in odd-numbered conditions and FR 40 in even-numbered conditions.

On choice trials, after the required number of responses on the center key, the center keylight was turned off and the two side keys were lit, with the left key green and the right key red. A single response on the left key constituted a choice of the standard alternative, and a single response on the right key constituted a choice of the adjusting alternative.

If the standard (left) key was pecked during the choice period, the right keylight was turned off. The left key remained green until the pigeon completed either an FR 10 response requirement (in Phase I) or an FR 1 response requirement (in Phase II). Then, the left keylight was turned off and grain was presented for 2 s. The white houselights were then lit and a 20-s ITI began.

If the adjusting (right) key was pecked during the choice period, the left keylight was turned off. The right key remained green until the pigeon completed the adjusting ratio requirement. After the pigeon completed the adjusting ratio, the right keylight was turned off and grain was presented for 6 s. The white houselights were then lit and a 20-s ITI began.

The adjusting-ratio procedure was the same as in Experiment 1. That is, each block of four trials consisted of two forced trials (with only the left or right key lit) followed by two choice trials. The adjusting ratio began at two responses in the first session of each condition, and it could be increased or decreased by two responses after each block of four trials.

Phases III and IV (Conditions 9–16)

The pre-trial schedule was FR 1 in odd-numbered conditions and FR 40 in even-numbered conditions. The procedure was the same as in Phases I and II until a choice response was made. If the left green key (the standard alternative) was pecked during the choice period, both keylights were turned off, there was a 2-s delay with the green houselights on, and then grain was presented for 2 s. If the right red key (the adjusting alternative) was pecked during the choice period, both keylights were turned off, there was an adjusting delay with the red houselights on, and then grain was presented for 6 s. The adjusting-delay procedure was the same as in Experiment 1. That is, the adjusting delay began at 0 s in the first session of each condition, and it could be increased or decreased by 1 s after each block of four trials.

The only difference between Phases III and IV was that the ITI (with the white houselights on) was 20 s in Phase III and 3 s in Phase IV.

Results and Discussion

Table 2 shows the number of sessions needed by each pigeon to reach the stability criteria in each condition. All data analyses were based on the results from the six half-session blocks that satisfied the stability criteria. For each pigeon and each condition, the mean adjusting ratio (in Phases I and II) or mean adjusting delay (in Phases III and IV) from these six half-session blocks was used as a measure of the indifference point. Pigeon P4 died before the start of Phase III, so only 7 pigeons completed Phases III and IV.

To summarize the results of this experiment, although the indifference points showed that the pigeons were sensitive to the standard and adjusting ratios, delays, and reinforcer amounts, no consistent effects of the pre-trial response requirement were found in any of the four phases. For each pigeon, Figure 3 presents the indifference points (mean adjusting ratios) from the eight conditions in Phases I and II. As with the rats in Experiment 1, the mean adjusting ratios were larger when the standard schedule was FR 10 (Phase I, M  =  29.2 responses) than when the standard schedule was FR 1 (Phase II, M  =  10.8 responses). These data show that the pigeons' choices were sensitive to the difference between 2-s and 6-s reinforcers (because the adjusting ratios were consistently greater than the standard ratios). However, there were no consistent differences between the conditions with pre-trial response requirements of FR 1 (white bars) and FR 10 (black bars). Examination of the results from individual pigeons suggests possible effects in Phase II, but the directions of the effects were different for different pigeons. For example, in Phase II, Pigeons P1 and P2 had larger adjusting ratios with the pre-trial FR 40, but Pigeons P4 and P7 showed the opposite effect—larger adjusting ratios with the pre-trial FR 1. Averaged across pigeons and replications, the mean adjusting ratios in Phase II were very similar with the pre-trial response requirements of FR 1 (M  =  11.3 responses) and FR 40 (M  =  10.2 responses).

Fig 3.

The mean adjusting ratios are shown for each pigeon in Experiment 2, from Phase I (where the standard schedule was FR 10) and Phase II (where the standard schedule was FR 1).

Figure 4 presents the indifference points (mean adjusting delays) for all the pigeons in Phases III and IV. Again, there were no consistent effects of the pre-trial FR in either phase. Only 1 pigeon (P6) showed consistently longer adjusting delays with the pre-trial FR schedule, and 2 pigeons (P5 and P8) showed the opposite pattern. (Note that these were not the same pigeons that showed possible effects of the pre-trial FR schedule in Phase II.) Averaged across subjects and replications, the mean adjusting delays were very similar with the short and long pre-trial ratio schedules. In Phase III, the group means were 5.2 s with FR 1 and 4.6 s with FR 40. In Phase IV, the group means were 4.0 s with FR 1 and 3.7 s with FR 40.

Fig 4.

The mean adjusting delays are shown for each pigeon in Experiment 2, from Phase III (where the ITI was 20 s) and Phase IV (where the ITI was 3 s).

In summary, the pigeons' indifference points showed that they were sensitive to the different reinforcer amounts, delays, and ratio requirements that were used as the standard and adjusting alternatives (because the adjusting ratios were greater than the standard ratios, and the adjusting delays, with only a few exceptions, were greater than the standard delay of 2 s). However, there were no systematic effects of the pre-trial response requirements in any of the four phases.

One possible explanation of the differences between the results from the pigeons and rats is that the times needed to complete the ratio requirements may have been different for the two species. To test this possibility, ratio completion times and response rates for the pigeons in this experiment and the rats in Experiment 1 were compared. In Phases I and II, the mean time to complete the pre-trial FR 40 was 19.5 s for the pigeons and 26.4 s for the rats. There was, however, considerable variability among subjects, and this difference was not statistically significant, t(10)  =  −1.25, ns. The time needed to complete the standard FR 10 schedule in Phase I was 3.0 s for the pigeons and 4.l s for the rats, and this difference was significant, t(10)  =  3.32, p < .01. The times needed to complete the adjusting ratio in Phases I and II depended, of course, on the current size of the ratio, so response rates were calculated instead. The mean response rates on the adjusting ratio were 3.3 responses/s for the pigeons, and 3.1 responses/s for the rats, t(10)  =  0.98, ns. Although the average response times tended to be slightly shorter for pigeons than for rats in Phases I and II, the results from Phases III and IV suggest that these time differences were not responsible for the performance differences between the species. Because delays, not ratio schedules, followed the choice responses in Phases III and IV, the only ratio schedule was the pre-trial FR 40. The mean times to complete the pre-trial FR 40 were very similar for the pigeons (M  =  24.1 s) and the rats (M  =  24.3 s), and yet Phases II and IV were where the clearest differences in the effects of the pre-trial ratio were found between the two species.

GENERAL DISCUSSION

The main purpose of these two experiments was to determine whether the size of a pre-trial response requirement would have any effect on the choice responses of rats or pigeons in a self-control choice situation. The results from the two species were quite different. In many of the conditions examined, when the pre-trial schedule was increased from FR 1 to FR 40, the rats' indifference points increased, indicating an increase in preference for the larger, more delayed reinforcer. In contrast, the pigeons did not show any systematic effects of the pre-trial ratio requirement in any of the conditions. We can now consider the implications of these results for molar and molecular theories of choice.

Beginning with the results from the pigeons, the absence of an effect of the pre-trial FR schedule is consistent with the previous findings of Mazur (1988), and it implies that the molar contingencies had no discernable control over the pigeons' choices. If a molar variable such as the amount of food per response (or the amount of food per unit of time) exerted some control over the animals' choices, preference for the adjusting alternative should have increased when the pre-trial ratio was FR 40. The fact that there was no such increase in preference seems to show that the pigeons' choices were based solely on the events that followed the choice response (the standard and adjusting delays, ratios, and food amounts), not on any events that preceded the choice response. This behavior is consistent with a molecular approach, such as the hyperbolic-decay model (Equation 1).

There are at least two different ways to interpret the behavior of the rats, which did show an effect of the pre-trial ratio. One possibility is that their choice behavior in this type of situation is jointly controlled by molecular variables (e.g., the delay between a choice response and food delivery) and molar variables (e.g., the food/response ratio for the two alternatives). The suggestion that both molecular and molar variables control choice behavior is not novel; it is represented in several mathematical models of choice (e.g., Grace, 1994; Killeen, 1982; Madden, Bickel, & Jacobs, 2000; Squires & Fantino, 1971). For example, in delay-reduction theory (Squires & Fantino, 1971), choice proportions in concurrent-chains schedules are determined by both the relative rates of primary reinforcement (a molar variable) and the reduction in delay to reinforcement signaled by the entry into a terminal link (a molecular variable).

Another possibility, as explained in the Introduction, is that a purely molecular approach can account for the effects of the pre-trial response requirement if the reinforcers delivered on several subsequent trials are included in the analysis. A choice of the standard alternative, with its shorter delay to food, meant that the food deliveries on all subsequent trials would occur sooner as well. However, this advantage of choosing the standard alternative was reduced when the pre-trial ratio was FR 40 rather than FR 1, because the extra time needed to complete the larger ratio increased the delays to the reinforcers delivered on all subsequent trials, so they should have less effect on the current choice.

To evaluate the plausibility of this account, computer simulations were used to obtain rough quantitative predictions from Equation 2 for the schedules used in Phases III and IV, where the largest effects of the pre-trial ratio was observed with the rats. To obtain these predictions, it was assumed that the rats responded on the FR 40 schedule at a rate of 2 responses/s (which is approximately what they did in Experiment 1), and that the reinforcers delivered on the subsequent 10 trials added to the values of both alternatives when making a choice on the current trial. Based on estimates from previous studies with rats (e.g., Green et al., 2004; Mazur, 2007; Mazur & Biondi, 2009; Richards et al, 1997), the decay parameter, K, was set at 0.2. With these parameters, Equation 2 predicted only a small effect of the pre-trial ratio for Phase III, in which the ITI was 20 s: The predicted indifference points were 8.0 s for a pre-trial ratio of FR 1, and 8.6 s for FR 40, which is a difference of only 7%. Because many factors can affect these predictions (including the value of K, the decision to include the following 10 trials, and the assumption that a 2-pellet reinforcer has exactly twice the value of a 1-pellet reinforcer), additional simulations were run using different parameter values and different assumptions. Naturally, the predicted indifference points varied with the changes in parameters, but in all cases, Equation 2 predicted an increase of about 5% to 10% as the pre-trial ratio was increased from FR 1 to FR 40 in Phase III. The actual indifference points for the rats in Phase III increased from a mean of 12.0 s with FR 1 to 19.6 s with FR 40, which is an increase of 63%. Furthermore, although there were large individual differences among the rats, all rats showed larger percentage differences than predicted by Equation 2 (with increases ranging from 49% to 85% for the 4 rats).

Using the same parameter values and assumptions for Phase IV (in which the ITI was decreased to 3 s), Equation 2 predicted indifference points of 5.4 s for a pre-trial ratio of FR 1, and 8.2 s for FR 40—an increase of about 50%. Again, similar percentage differences were predicted when different parameter values and assumptions were used in the simulations. The actual indifference points for the rats in Phase IV increased from a mean of 4.6 s with FR 1 to 15.5 s with FR 40, which is an increase of over 300%. All rats showed larger percentage increases than predicted by Equation 2 (ranging from 55% to 669% for the 4 rats).

Based on these (admittedly rough) simulations, it appears that the differences obtained from the rats between the FR-1 and FR-40 conditions in Phases III and IV were too large to be accounted for by taking into account the reinforcers delivered on subsequent trials in Equation 2. I could find no reasonable parameter values that led to predicted differences as large as those actually observed. However, it is still possible that these results could be explained by a molecular approach that was based on different assumptions (e.g., a model that treated the total time from the start of the pre-trial ratio until the delivery of food as the “delay” to reinforcement, even though the choice response was made after the pre-trial ratio was completed). To summarize, the computer simulations suggest that Equation 2, as it has normally been applied, cannot account for the magnitude of the effect observed with the rats. Therefore, either a molecular model based on different assumptions or a model that includes the influence of molar variables such as overall rate of reinforcement seems to be needed to account for the behavior of the rats.

There seems to be a fairly straightforward empirical test that could be conducted to determine whether a model that includes molar variables is needed to account for the rats' results. In the present research, the ITI was kept constant (at either 20 s or 3 s) for both the standard and adjusting trials. If the ITI itself were adjusted to keep the time between trials constant regardless of which choice an animal made (as is done in many experiments on self-control choice), the effects of reinforcers on future trials would cancel out in Equation 4, and this equation would therefore predict that the size of the pre-trial ratio should have no effect on choice. If equating the time between choice responses eliminated the effect of the pre-trial ratio, this would imply that no molar variable need be considered. Conversely, if the effect of the pre-trial ratio remained, this would strongly suggest that some sort of molar analysis was warranted.

The fact that no such effect of the pre-trial ratio was found with the pigeons in Experiment 2 does not necessarily imply that there is a fundamental species difference in how rats and pigeons perform in this choice situation. In some previous work with the adjusting-delay procedure, I found an apparent difference between these two species in how variations in the ITI affected choices involving probabilistic reinforcers (Mazur, 2005, 2007)—rats' choices were affected by ITI duration whereas pigeons' choices were not. However, later research by Mazur and Biondi (2011) showed that this apparent species difference could be eliminated by making some changes in the procedures used for assessing choice. It is therefore quite possible that pigeons might also show an effect of pre-trial response requirements if different schedule values or other procedural changes were tried.

In summary, the main finding of these experiments was that rats' choices were affected by the pre-trial ratio requirement, but pigeons' choices were not. Although the exact reasons for the rats' performance cannot be determined from the present data, the finding of an effect of the pre-trial response requirement is important for both theoretical and practical reasons. At a theoretical level, it suggests either that molar variables (e.g., the overall rates of reinforcement) can affect self-control choices, or that alterations in molecular models such as the hyperbolic-decay model may be needed to account for this effect. At a practical level, there has been intense interest both in factors that affect self-control choices (e.g., Madden & Bickel, 2010; Odum, 2011) and in finding ways to increase self-control choices by people in everyday situations (e.g., Bickel, Yi, Landes, Hill, & Baxter, 2011; Black & Rosen, 2011; Dixon & Holcomb, 2000; Reynolds, 2006; Schweitzer & Sulzer-Azaroff, 1988). The evidence that a larger pre-trial response requirement leads to less impulsive choices (at least for rats) could suggest ways to decrease impulsive behavior in humans as well. Given the interest in finding ways to increase self-control choices, additional research on the effects of pre-choice response requirements could be valuable.