Effects of a History of Differential Reinforcement on Preference for Choice

Abstract

The effects of a history of differential reinforcement for selecting a free-choice versus a restricted-choice stimulus arrangement on the subsequent responding of 7 undergraduates in a computer-based game of chance were examined using a concurrent-chains arrangement and a multiple-baseline-across-participants design. In the free-choice arrangement, participants selected three numbers, in any order, from an array of eight numbers presented on the computer screen. In the restricted-choice arrangement, participants selected the order of three numbers preselected from the array of eight by a computer program. In initial sessions, all participants demonstrated no consistent preference or preference for restricted choice. Differential reinforcement of free-choice selections resulted in increased preference for free choice immediately and in subsequent sessions in the absence of programmed differential outcomes. For 5 participants, changes in preference for choice were both robust and lasting, suggesting that a history of differential reinforcement for choice may affect preference for choice.

Humans and nonhumans have been shown to prefer stimulus arrangements with more alternatives over otherwise comparable stimulus arrangements with fewer alternatives (e.g., Catania, 1975, 1980; Catania & Sagvolden, 1980; Tiger, Hanley, & Hernandez, 2006; Voss & Homzie, 1970). The cause or causes of this preference remain unknown, but it is plausible that both phylogenic and ontogenic factors play a role (Catania, 1980). That is, preference for choice (i.e., stimulus arrangements with more alternatives) could have survival value and therefore be selected through natural or cultural selection. For example, a preference for access to more sources of food or water versus fewer sources could prove advantageous in the event that one or more sources cease to provide adequate supplies. Additionally, preference for choice could be conditioned within an individual's lifetime through differential reinforcement (e.g., qualitatively or quantitatively greater reinforcement corresponding to stimulus arrangements with more alternatives).

The effects of this history of differential reinforcement could be extended through the process of generalization to situations in which the stimulus arrangement with more alternatives does not provide qualitatively or quantitatively greater reinforcement. For example, Tiger et al. (2006, Experiment 1) used a concurrent-chains arrangement to evaluate preference for choice in humans. In their experiment, 6 preschool children made selections from three colored worksheets corresponding to a “choice,” “no choice,” or control terminal link. Completion of the choice terminal link resulted in praise and the presentation of a plate with five identical reinforcers from which the participant selected one reinforcer. Completion of the no-choice terminal link resulted in praise and the presentation of a plate with one reinforcer identical to the reinforcers in the choice terminal link. Completion of the control terminal link resulted in praise. Three children consistently selected the choice arrangement and one consistently selected the no-choice arrangement. Two children initially demonstrated a preference for the choice arrangement, but showed little difference between the choice and no-choice arrangements in later sessions. For the children who preferred the choice arrangement, Tiger and colleagues hypothesized that the multiple identical items in this arrangement may have served as an “illusory discriminative stimulus.” That is, an extraexperimental history of stimulus arrangements with more alternatives resulting in qualitatively or quantitatively greater reinforcement may have generalized to the choice arrangement in the experimental preparation (through stimulus generalization), even though there was no quantitative or qualitative difference in reinforcement outcome between the choice and no-choice arrangements.

However, Schmidt, Hanley, and Layer (2009) demonstrated that stimulus generalization related to the number of stimuli is not necessary to explain the preference for choice. They used a similar arrangement as Tiger et al. (2006) but controlled for the number of stimuli by presenting the same number of stimuli in both the choice and no-choice arrangements. Instead, the choice and no-choice arrangements were distinguished by a picture of a hand pointing to the child and the child selecting one item from the array of five identical items (choice) and a picture of a hand pointing to the experimenter and the experimenter selecting one item from the array of five identical items and presenting the item to the child (no-choice). Under this preparation, 5 of 6 participants demonstrated a preference for the choice arrangement, with the remaining participant responding indiscriminately (i.e., selecting the control arrangement with access to verbal praise only as often as any of the others).

The findings by Schmidt et al. (2009) may be interpreted as a demonstration that stimulus generalization based on the number of alternatives may be sufficient, but not necessary, to explain the preference for choice in preparations that provide no differential reinforcement for selecting the choice arrangement.1 If this is true, then additional functional relations are indicated. The process of conditioning offers another possibility. In a given situation, an act of choosing may be followed by qualitatively or quantitatively greater reinforcement. As an increasing number and forms of acts of choosing are followed by differential reinforcement, it becomes increasingly likely that the act of choosing will become a generalized response followed by qualitatively and quantitatively greater reinforcement. Thus, through repeated occurrences of choosing followed by reinforcement, the opportunity to choose may become a conditioned reinforcer (Fisher, Thompson, Piazza, Crosland, & Gotjen, 1997). At least two variables are critical in this analysis: the history of reinforcement and the different choosing responses that have been reinforced. The latter variable determines the extent of the former, as reinforcing varied choosing responses increases the probability that different choice responses will occur in novel arrangements (Stokes & Baer, 1977).

In summary, people may demonstrate a preference for choice in the absence of programmed differential outcomes when choice consists of more alternatives (e.g., Tiger et al., 2006) and when choice consists of the opportunity to choose (e.g., Schmidt et al., 2009). It is possible that a nonexperimental history of differential reinforcement may explain, at least in part, this preference for choice. Indeed, a history of differential reinforcement for choice selections is often posited as a possible explanation for preference for choice (e.g., Catania, 1975, 1980), but to date there has been no direct experimental evaluation of this explanation. Therefore, the purpose of the present study was to determine the effects of a history of differential reinforcement for choice upon the preference for choice in the absence of programmed differential outcomes between choice and no-choice arrangements.

In the present study, we evaluated preference for choice in the context of a computerized game of chance. Participants who showed no preference for choice in initial sessions were selected. After baseline sessions, selections for a “free-choice” stimulus arrangement over a “restricted-choice” stimulus arrangement were differentially reinforced with points. In the free-choice arrangement, participants selected three numbers, in any order, from an array of eight numbers presented on the computer screen. In the restricted-choice arrangement, participants selected only the order of three numbers that were preselected by the computer program from the array of eight numbers. Maintenance of the preference for the free-choice arrangement was then assessed in the absence of differential reinforcement and in the presence of differential reinforcement for the restricted-choice arrangement.

METHOD

Participants

Eleven undergraduate students enrolled in an introductory psychology course at a small New England college participated in this experiment. These participants ranged in age from 18 to 21 years old, and 9 of the participants were males. Participants received extra credit for their participation in this study, but were also given a comparable option (i.e., a brief paper) for obtaining the equivalent amount of extra credit. IRB approval was obtained and informed consent procedures were followed for each participant. Within the first two sessions, 4 participants did not meet criteria for inclusion and were removed from the experiment.

Setting and Materials

Sessions were conducted in a room equipped with two tables, chairs, and several laptop computers. The computers were loaded with a game programmed into Powerpoint®. Each participant was provided with headphones and a computer mouse. The headphones did not mask all other sound in the room but kept any auditory feedback from the game isolated to the intended participant. From 1 to 6 participants were in the room running sessions at the same time. During sessions, participants did not interact with one another, and all participants were instructed not to discuss the experiment with anyone except the experimenters. An experimenter (the third author) was in the room during the session but sat at a table away from the participants and read or worked quietly while the participants played the game. On some occasions, she left the room for brief periods.

Procedure

A concurrent-chains arrangement was used to measure preference for a free-choice arrangement versus a restricted-choice arrangement. Trials were presented via a computer-based game that consisted of three sets of 40 trials. The first 32 trials of each set were exposure trials. During exposure trials the participant was presented with either a free-choice initial link, in which a gray message box appeared on the screen with the words “You Select” inside (see Figure 1, top panel), or a restricted-choice initial link, in which a gray message box appeared on the screen with the words “Numbers Generated” (see Figure 1, bottom panel). Initial link presentations were alternated every trial. Thus, in each set of 32 exposure trials, the initial link for free choice was presented in a total of 16 trials and the initial link for restricted choice was presented in a total of 16 trials. A single response (i.e., clicking on the gray message box using a computer mouse) in the initial link produced the corresponding terminal link. In the free-choice terminal link, an array of eight numbers was presented on the screen and the participant clicked on any three of the numbers to enter them into a blue game box (see Figure 2, top panel) and then clicked “Ready.” In the restricted-choice terminal link, eight numbers were presented, but five numbers (randomly determined by the program each trial) were dimmed and inoperative. The participant completed this terminal link by clicking on the three operative numbers, thereby entering them into a blue game box, and then clicking “Ready” (see Figure 2, bottom panel). Reinforcement (point delivery) was provided on a variable-ratio (VR) schedule of terminal-link completions. The specific schedule varied according to the condition of the experiment.

Figure 1.

Top panel: Schematic representing the free-choice initial link during an exposure trial. Bottom panel: Schematic representing the restricted-choice initial link during an exposure trial.

Figure 2.

Top panel: Schematic representing the free-choice terminal link. In this schematic, the participant has clicked on the number 2 and has two more numbers to select. His points earned going in to this trial is 3. Bottom panel: Schematic representing the restricted-choice terminal link. In this schematic, the participant has clicked on the number 2 and has two more numbers to click on (“5” and “7”). The participant had 3 points earned going in to this trial.

The last eight trials of each set consisted of choice trials. During these trials, both response options were present on the screen. The response options were presented either left-to-right, or top-to-bottom (see Figure 3). The position of each response option did not change within sets of choice trials but was systematically altered across sets. Clicking on a response option immediately produced the corresponding terminal link. Terminal links were identical to the terminal links in the exposure trials, except that reinforcement was delivered on a random-ratio (RR) schedule corresponding to the VR schedule in the preceding exposure trials. RR schedules were used for choice trials because the number of exposures to free-choice and restricted-choice terminal links could not be controlled during choice trials. A running tally of all points earned was visible each time a terminal link was presented. In summary, preference for free choice versus restricted choice was assessed using a concurrent-chains arrangement, with an FR-1 schedule during initial links and an RR schedule for completion of terminal links.

Figure 3.

Top panel: schematic representing horizontal placement of initial links during choice trials. The positions of initial links were alternated across sessions. Bottom panel: schematic representing vertical placement of initial links during choice trials. The positions of initial links were alternated across sessions.

Response Measurement

The computer program recorded all mouse clicks to the initial and terminal links and all points delivered by trial on an Excel® spreadsheet.

Preference was calculated from the choice trials by subtracting the proportion of responses to the initial link corresponding to restricted choice from the proportion of responses to the initial link corresponding to free choice. The resulting number (hereafter referred to as the “choice quotient”) was a number from −1 to 1 (inclusive), with −1 indicating exclusive responding to the restricted-choice initial link during choice trials, 0 indicating indifference between free-choice and restricted-choice links during choice trials, and 1 indicating exclusive responding to the free-choice initial link during choice trials.

As there were 24 choice trials in each session, the number of exposures to each terminal link could vary within and across sessions, possibly resulting in different reinforcement histories across the two terminal links even when no differences were programmed. Therefore, two measures of obtained points following terminal links were calculated. The relative points delivered following each terminal link each session was calculated by subtracting the proportion of points delivered following the restricted-choice terminal links from the proportion of points delivered following the free-choice terminal links. The resulting number (hereafter referred to as the “relative points quotient”) was a number from −1 to 1 (inclusive), with −1 indicating that all of the points were delivered following restricted-choice terminal links, 0 indicating that points were delivered evenly across terminal links, and 1 indicating that all of the points were delivered following free-choice terminal links. Two measures were calculated—one using points delivered across all trials in a session (relative points quotient–all trials) and the other using only points delivered following choice trials (relative points quotient–choice trials).

Also, as the reinforcement schedules during choice trials were RR schedules, it was possible for obtained schedules of reinforcement to differ from programmed schedules of reinforcement. Therefore, the relative probability of point delivery corresponding to each terminal link each session was calculated by dividing the number of points delivered following the restricted-choice terminal link by the number of presentations of the restricted-choice terminal link and subtracting this number from the number of points delivered following the free-choice terminal link divided by the number of presentations of the free-choice terminal link. The resulting quotient (hereafter referred to as the “relative odds quotient”) was a number from −1 to 1 (inclusive), with −1 indicating that all of the restricted-choice terminal links and none of the free-choice terminal links were followed by a point, 0 indicating that the probability of point delivery following restricted-choice terminal links was the same as the probability of point delivery following free-choice terminal links, and 1 indicating that none of the restricted-choice terminal links and all of the free-choice terminal links were followed by a point.

In summary, the choice quotient indicates which initial link was selected more frequently during choice trials, the relative points quotient indicates which terminal link was followed by more points during all trials and during choice trials (two calculations), and the relative odds quotient indicates which terminal link was correlated with better odds of point delivery. Except at the extreme values, the relative points and the relative odds quotients do not provide actual points or probabilities in any given session (these data are available from the first author); rather, these measures indicate which terminal link had the most points or the best probability of being followed by a point within a session.

In addition to the above measures, the specific numbers selected or generated per trial were also recorded. Finally, the latency from presentation to criterion of the terminal links (i.e., the time to select numbers during free-choice terminal links or to enter generated numbers during restricted-choice terminal links) was measured each trial in order to rule out changes in preference due to differences in effort or delays to reinforcement. There were no noteworthy findings related to the specific numbers the participants selected or the numbers that were generated by the computer; the latencies of terminal links are presented with the other results below.

Participant Selection

Only participants who did not demonstrate a preference for free choice during initial baseline sessions were eligible for this experiment. These 7 participants were assigned to one of four groups corresponding to different legs of a multiple-baseline-across-participants design. Assignment was based upon schedule availability. Each group differed in the number of baseline sessions (1, 2, 3, or 4) run before implementation of the intervention.

Sessions and Instructions

Immediately prior to each session, the experimenter loaded the designated version of the computer program for each participant onto each participant's computer and opened the program. When the participants entered the room they were shown to their computer and instructed to begin the game whenever they were ready. During the first session with each participant, the experimenter read the instructions aloud as they appeared on the screen and asked the participant if he or she had any questions. The instructions included the following:

Shortly, you will be presented with a game. Your goal is to win points and have fun. At the beginning of each turn, you will see one or two buttons. Clicking on a button will advance you to the game screen. If you click “You Select” then in the upcoming turn you will need to select three numbers to play the game. If you click “Numbers Generated” then in the upcoming turn you will need to enter three computer-generated numbers to play the game. If you see only one button on the screen, you will need to select this button to play the game. If you see two buttons, you may select either button.

When you get to the game screen, you will see 8 buttons along the lower center screen, each with a number from 1 to 8 on it. If you are selecting numbers (“You Select”), then you play the game by entering the numbers you want by navigating your cursor over a number and left-clicking it. Numbers can be entered in any order, and each number can be entered only once per turn. If you are entering numbers generated for you (“Numbers Generated”), then you play the game by clicking on the indicated numbers. Clicking on other numbers will do nothing. Numbers can be entered in any order, and each number can be entered only once per turn.

When all of the numbers are entered on the game screen, clicking the “Ready” button will generate up to three winning numbers. You will not see these numbers, but they will automatically be compared to the numbers you entered. If any of the numbers you entered matches a winning number, you will see the words “You Matched!”, hear a pleasant tone, and win a point. Your points will be tracked for you in a box in the upper right corner of the screen. As soon as one of the winning numbers matches one of the numbers you entered, the computer stops generating winning numbers. The most you can win each turn is one point. If none of the numbers you entered matches a winning number, you will see the words “No matches” and hear a different tone. You will not earn a point that turn.

….When you are ready to begin, click the ‘start’ button below using your mouse.

After the initial session, the experimenter did not read the instructions with the participant. When the participant came to the last screen of instructions and clicked on “Ready,” a message box appeared and prompted him to click “okay” if he was ready to begin the game, or to click “cancel” if he wanted to read the instructions again. Once the participant clicked “okay,” the game began.

The participant was then presented with a total of 120 trials (described earlier). After the last trial, the instructions on the screen prompted the participant to inform the experimenter that he was done. Sessions typically lasted 10 to 20 min, and one to two sessions were run per day, with a minimum of 5 min separating each session.

Equal sessions

During these sessions, the participant had an equal probability of winning a point in the free-choice and restricted-choice terminal links. The schedule of reinforcement in the free-choice and restricted-choice terminal links was VR 2 during exposure trials and RR 2 during choice trials. As the experiment was designed to replicate a game of chance, specific schedules are labeled according to the odds of winning programmed for each terminal link, expressed as ratio x∶y, where x is the mean number of programmed points delivered out of y opportunities (y is kept constant at “8”—the number of trials within each choice set—for ease of comparison within and across schedules). The terminal link with the highest odds of winning is listed first; if the probabilities are equal, the FC terminal link schedule is listed first. Using this convention, the label of the schedules in equal sessions is FC 4∶8 RC 4∶8. See Table 1 for the reinforcement schedules used in each condition and their corresponding designations.

Table 1.

Schedules of Reinforcement for Exposure and Choice Trials.

Differential reinforcement of free choice (DR FC) sessions

In this condition, the reinforcement schedules for free- and restricted-choice selections were adjusted in order to create a history of differential reinforcement for free choice. In all but one of the DR FC sessions for one participant, the schedule of reinforcement was FC 7∶8 RC 2∶8. In the last DR FC session with Participant DK, the schedule of reinforcement was FC 7∶8 RC 0∶8.

Differential reinforcement of restricted choice (DR RC) sessions

This contingency-reversal condition was implemented to replicate the effects of differential reinforcement and to test the durability of preference for free choice under conditions in which differential reinforcement favored restricted choice. For most DR RC sessions, the schedule of reinforcement was RC 7∶8 FC 2∶8; for one session with Participant BE the schedule of reinforcement was RC 7∶8 FC 0∶8.

Experimental Design and Sequence of Conditions

A multiple-baseline-across-participants design was used to demonstrate experimental control. As described previously, participants were divided into four groups, with each group exposed to one, two, three, or four baseline sessions. Following baseline sessions, two sessions of DR FC were run (three sessions for DK). All participants were then returned to equal schedules in order to assess any changes in preference. Subsequent schedule manipulations were made on an individual basis and consisted of (a) DR RC, in which participants were exposed to two sessions of DR RC, followed by at least one equal session (Participants AB, AD, BC, and BE), (b) a “booster” exposure to DR FC followed by at least one equal session (Participants CG and DK). No further manipulations were conducted with Participant CJ, who stopped attending sessions after Session 10.

Program Variations

The game was programmed using Visual Basic within Powerpoint®. Multiple variations of the basic program were created in order to accomplish several objectives, including: (a) the appearance of randomness in point acquisition during exposure trials, (b) creation of different schedules of reinforcement for completion of terminal links, (c) identification of potential position biases, (d) control for potential sequence effects (i.e., counterbalancing). During pilot work before the current study, the variations were tested with 3 adults naïve to the purposes of the study to determine appropriate session length and to examine potential position bias and sequence effects. Further information on these variations is available from the first author.

RESULTS

Choice, relative points, and relative odds quotients for each session are depicted in Figure 4 (for Participants AB, AD, BC, and BE) and Figure 5 (for Participants CG, CJ, and DK). The choice quotients across choice sets for each participant are depicted on Figure 6 (a choice set consists of the eight choice trials following 32 exposure trials; there were three such sets within each session). Each of the 7 participants met criterion for inclusion by demonstrating no preference for free choice in the initial equal sessions. Four participants (AD, BC, BE, and CJ) showed a preference for restricted choice, whereas 3 participants (AB, CG, and DK) did not show a strong preference for either free or restricted choice. In these and subsequent equal sessions, all the participants won more points following the terminal link they selected most often due to the increased number of opportunities to win following this link (but see Session 5 for BC).

Figure 4.

Choice, relative points, and relative odds quotients for Participants AB, AD, BC, and BE under equal schedules of reinforcement (E), differential reinforcement of free choice (DR FC), and differential reinforcement of restricted choice (DR RC).

Figure 5.

Choice, relative points, and relative odds quotients for Participants CG, CJ, and DK under equal schedules of reinforcement (E), differential reinforcement of free choice (DR FC), and differential reinforcement of restricted choice (DR RC).

Figure 6.

Choice quotient across choice sets under equal schedules of reinforcement (E), differential reinforcement of free choice (DR FC), and differential reinforcement of restricted choice (DR RC). There were three choice sets in each session.

Following baseline sessions, participants were exposed to two DR FC sessions, which resulted in increases in the preference for free choice for all participants (see Figure 6 for a depiction of choice quotients across choice sets). Visual inspection indicated a relatively low change in level for DK; therefore an extra DR FC session was run for this participant. During each DR FC session, all participants won more points and had better odds of winning following free-choice selections, as indicated by the relative points quotients and relative odds quotients in Figures 4 and 5.

With all participants, a return to equal schedules was then implemented in an attempt to detect the effects of a recent history of differential reinforcement of free choice. For 5 of the 7 participants, preference for free choice remained well above baseline levels until differential reinforcement of restricted choice was implemented (AB, AD, BC, BE) or until the participant no longer attended sessions (CJ). Visual inspection of responding across choice sets revealed no indication that preference was changing for any of these 5 participants in this condition.

For 2 of the 7 participants, the effects of differential reinforcement of free choice were transient. CG's preference for free choice dropped to predifferential reinforcement levels during the second session following differential reinforcement of free choice, and two more DR FC sessions were implemented. Following this “booster” intervention, CG demonstrated a strong preference for free choice until the end of the experiment (nine more sessions). In the third session following DR FC sessions, DK demonstrated a slight preference for restricted choice, and one more DR FC session was run. Although DK consistently selected the free-choice initial links during choice trials in this booster session, he did not demonstrate a preference for choice in the subsequent equal sessions.

For all but one participant differential reinforcement of free choice resulted in a durable preference for free choice under subsequent equal sessions. But how durable was this preference? One test of durability is to measure performance over time. This test was conducted with CG and CJ to the extent possible given participant availability. CJ demonstrated a strong preference for free choice during all five equal sessions following DR FC sessions, and CG showed a consistent preference for free choice in the nine equal sessions (216 choice trials) following the booster DR FC sessions.

Another test of durability is to measure performance under conditions favoring an alternative. Accordingly, the restricted-choice selections of 4 participants (AB, AD, BC, and BE) were differentially reinforced. These participants demonstrated a preference for restricted-choice terminal links during both DR RC sessions (AD, BC), or by the second DR RC session (AB, BE). BE demonstrated a strong and stable preference for free choice during the first DR RC session, and therefore in the second DR RC session reinforcement was completely removed from the free-choice terminal link. None of the 4 participants exposed to DR RC demonstrated a strong preference for free or restricted choice following DR RC sessions. It is possible that differential reinforcement of restricted choice would have resulted in a preference for restricted choice in subsequent equal sessions had there been no prior differential reinforcement of free choice, but this cannot be confirmed with the current experimental arrangement.

Finally, the latencies from presentation to criterion during restricted-choice terminal links were consistently greater than the latencies during free-choice terminal links. This was true across sessions: Mean latencies of restricted-choice terminal links exceeded latencies of free-choice terminal links by at least 0.1 s or more in 69 of the 86 sessions. This was also true across conditions, as shown in Table 2: For each participant, the mean latencies of restricted-choice terminal links were greater than the latencies of the free-choice terminal links within each condition, with only a few exceptions.

Table 2.

Mean latencies per trial from presentation to criterion (in seconds) during free-choice and restricted-choice terminal links measured across phases.

DISCUSSION

Our results support the claim that a history of differential reinforcement may affect preference for choice (e.g., Catania, 1980). In particular, a history of differential reinforcement may affect preference for choice in two ways. First, differential reinforcement for free choice in one context may affect preference for choice in that same context even when there are no longer differential outcomes (i.e., maintenance). Second, a history of differential reinforcement for free choice may affect responding in new contexts in which there are no differential outcomes (i.e., generalization). As far as we know, this experiment provides the first laboratory demonstration of the maintenance of preference for free choice following differential reinforcement of free choice.

The results of this experiment are clear, but not unqualified. Isolating choice as a variable for investigation presents at least three challenges for researchers. First, the value of reinforcement must be held constant across the choice and no-choice arrangements. If, for example, the choice arrangement provides access to a more preferred reinforcer or a reinforcer of even slightly greater magnitude than the no-choice arrangement, then the isolation of choice as a variable is compromised. Researchers have addressed this challenge by using only highly preferred stimuli in both choice and no-choice arrangements (e.g., Smith, Iwata, & Shore, 1995), yoking reinforcement in the no-choice arrangement to reinforcement in the choice arrangement (e.g., Fisher et al., 1997), and using concurrent-chains arrangements with identical reinforcement (e.g., Tiger et al., 2006; for a discussion of the advantages and disadvantages of these procedures, see Fisher et al., 1997 and Tiger et al., 2006). A strength of the current study is the use of points as reinforcement for completion of both the free-choice and restricted-choice terminal links (e.g., Suzuki, 2000). Points do not vary across physical dimensions that may influence responding, and changes in the value of points due to motivational operations should affect responding equally across the free- and restricted-choice terminal links.

A second challenge in isolating choice as a variable is controlling for differences in the stimulus arrangements of the choice and no-choice conditions. We attempted to keep the free-choice and restricted-choice terminal links as equivalent as possible across physical dimensions (e.g., size of stimuli, shape of stimuli, colors, etc.) and across the effort required to complete each terminal link. Accordingly, eight numbers were presented in each terminal link and the same response effort (clicking three of the numbers and “Ready”) was required to complete each link. Still, the consistently shorter latencies in the free-choice terminal links suggest that participants may have developed more efficient responding in the free-choice terminal link (i.e., shorter latencies may have reduced the time to reinforcement) which in turn may have influenced preference. However, 3 participants (AB, BE, and CJ) demonstrated shorter latencies in free-choice terminal links and a mild or strong preference for restricted choice during equal sessions prior to DR FC sessions. Further, of the 4 participants exposed to DR RC sessions, all demonstrated shorter latencies in the free-choice terminal links in the equal session following DR RC, but 2 (AD, BC) showed a mild preference for free choice and the remaining 2 (AB, BE) showed a mild preference for restricted choice. Thus, shorter latencies in free-choice terminal links were not consistently correlated with preference for free choice in equal sessions.

A third challenge in isolating choice as a variable is controlling the exposure to the choice and no-choice terminal links and the subsequent in-session reinforcement histories. Catania (1975) addressed this challenge, at least in part, by using a concurrent VI schedule for the initial links leading to the terminal links. Whereas ratio schedules under concurrent schedules are more likely to produce exclusive responding, interval schedules are likely to produce less discrepancy between exposures to the terminal links. Our study employed FR schedules for the initial link rather than VI schedules, but we attempted to minimize differential exposure through a rich ratio (4∶1) of exposure trials to choice trials. Future researchers may consider employing a brief interval schedule in the initial links to further minimize differential exposure to the different terminal links.

Our data show that following differential reinforcement of free choice, 5 of 7 participants exhibited a strong and durable preference for free choice in subsequent equal sessions. However, when 4 of these participants were then exposed to differential reinforcement of restricted choice, none of them exhibited a strong or durable preference for restricted choice in subsequent equal sessions. This finding is not entirely unexpected, given the relatively recent histories of differential reinforcement for free choice and the differential exposure to free-choice terminal links in equal sessions following DR FC sessions (see Mazur, 1996). However, the lack of any strong or durable preference following DR RC sessions leaves open the possibility that there may be some phylogenic or ontogenic factors other than recent reinforcement histories influencing preference for free choice. Future research should examine the differential reinforcement histories necessary to produce a durable preference for free versus restricted choice.