Matching-To-Sample by Pigeons: The Dissociation of Samples from Reinforcement Contingencies

Abstract

It has been proposed that comparison choice in matching-to-sample should depend on two factors, the relative probability of reinforcement associated with each of the comparison stimuli and the conditional probability of each comparison stimulus being correct given presentation of one of the samples. DiGian and Zentall (in press) have shown that sample frequency together with the probability of choosing each of the comparison stimuli in training can influence comparison choice when delays are introduced, when the number of reinforcements associated with each of the comparison stimuli is equated. Furthermore, Clement and Zentall (2002) have found that sample frequency can affect comparison choice when delays are introduced independently of the number of choices of each of the comparison stimuli in training and the number of reinforcements associated with each of the comparison stimuli is equated. In the present experiment we found that the probability of choosing each of the comparison stimuli in training can affect comparison choice when delays are introduced, independently of sample frequency and when the number of reinforcements associated with each of the comparison stimuli is equated. Together, these experiments suggest that when the sample is not available, there is a partial dissociation between comparison choice and the probability of reinforcement associated with each of the comparison stimuli.

1. Introduction

The assessment of memory mechanisms in pigeons has made extensive use of matching-to-sample procedures. In matching-to-sample, the identity of the initial or sample stimulus indicates which of (typically) two test or comparison stimuli is correct (Skinner, 1950). One can think of matching-to-sample as a chained schedule in which responses to the sample produce the comparison stimuli and choice of the correct comparison stimulus results in reinforcement. Hartl and Fantino (1996) have proposed that in a matching task, comparison choice should depend on two factors, the relative probability of reinforcement associated with each of the comparison stimuli and the conditional probability of each comparison stimulus being correct given presentation of one of the samples.

At the start of training, prior to the association of the sample with the correct comparison stimulus, the probability of each of the comparisons being correct is typically .50. As the pigeon associates the sample with the correct comparison stimulus, the probability of being correct increases to almost 100%. Memory for the sample can then be tested by inserting a delay between the offset of the sample and the onset of the comparisons. The typical finding is that as the delay increases, memory for the sample declines and matching accuracy declines to near chance levels. If over trials the two samples are presented equally often and the two comparison stimuli are correct equally often, one would expect that the slopes of the retention functions would decline similarly with increasing delay (see Grant 1991; White and Wixted, 1999). Stated differently, as sample memory decreases with increasing delay, pigeons should choose comparisons on the basis of the relative probability with which they had been reinforced in training (McCarthy and Davison, 1980, 1991; White and Wixted, 1999). In support of this hypothesis, White (2001) has shown that when the conditional probability of reinforcement is varied as a function of the delay, pigeons are sensitive to this change. Similarly, Goodie and Fantino (1995, 1996) have shown that humans are sensitive to manipulation of the probability of reinforcement for comparison choice.

These earlier experiments have shown that organisms are sensitive to the differential probability of reinforcement for comparison choice under a variety of conditions but they have not addressed the question of whether variables such as sample frequency and comparison response frequency, in the absence of differential reinforcement frequency, might affect comparison choice. In most cases, the probability of reinforcement for comparison choice, the probability of sample presentation, and the probability of a response to each of the comparison stimuli in training are the same (.50). When they have been manipulated, either all three have been confounded (e.g., McCarthy and Davison, 1980) or sample frequency has been held constant (e.g., McCarthy and Davison, 1991).

We have tried to address the role of sample frequency and comparison choice frequency in a series of experiments (DiGian and Zentall, in press; Zentall and Clement, 2002). DiGian and Zentall (in press, Experiment 1) approached this question by manipulating sample frequency and the probability of reinforcement for a correct comparison response in such a way that the probability of reinforcement associated with each of the comparison stimuli was the same. For example using a red-green identity matching task, 80% of the trials began with a red sample and on those trials correct choice of the red comparison stimulus was reinforced 25% of the time (on those trials choice of the incorrect green comparison stimulus was never reinforced). The remaining 20% of the trials began with a green sample and on those trials correct choice of the green comparison stimulus was reinforced 100% of the time (on those trials choice of the incorrect red comparison stimulus was never reinforced). With this procedure, by the end of training, the pigeons were choosing the red comparison stimulus more often than the green comparison but they were receiving approximately 20 reinforcements each for choice of the red and green comparison stimuli. Following training with no delay, to induce errors, delays were inserted between the offset of the sample and the onset of the comparison stimuli. If, in the absence of memory for the sample, the probability of reinforcement associated with each of the comparisons controlled comparison choice, as predicted by White and Wixted (1999), the retention functions should have been similar in slope. On the other hand, if under these conditions comparison choice is controlled by the probability of reinforcement given a response there should be a bias to choose the green comparison. That is, at the end of training, there would have been many more nonreinforced responses made to the red comparison than to the green comparison. Finally, it is possible that comparison choice would be controlled by the overall sample probability. That is, if 80% of the trials in training involved a red sample, it would be most probable that the forgotten sample had been red and thus more probable that the correct response would be to the red comparison. This hypothesis would lead one to predict that in the absence of memory for the sample, there would be a bias to choose the comparison associated with the more frequent, red sample.

The results of this experiment showed that with increasing delay, the pigeons developed a strong bias to choose the comparison associated with the more frequent sample. Thus, it appears that the pigeons’ choice behavior was not influenced by the probability of reinforcement associated with each of the comparison stimuli (given a response). Furthermore, the number of reinforcements associated with each of the comparison stimuli did not control comparison choice either. Rather, comparison choice appears to have been controlled by the relative frequency with which each of the samples was presented.

Alternatively, choice of the comparison stimuli may have been influenced by the probability of making a particular comparison response. That is, the pigeons may have adopted a kind of single-code/default response strategy, such as ‘choose the red comparison unless the sample is green.’ On green sample test trials, as memory for the green sample is lost, the pigeons should begin to choose the red comparison. In contrast, on red sample test trials, the pigeons would tend to continue to choose the red comparison even in the absence of memory for the red sample. A follow-up experiment showed a similar effect when the red and green comparison stimuli were replaced by vertical lines and horizontal lines. Thus, the asymmetrical retention functions could not be attributed to the identity relation between the sample and comparisons (DiGian and Zentall, in press, Experiment 2). Finally, a third experiment demonstrated that the asymmetrical retention functions could not be attributed to differential intertrial interference resulting from the higher probability that the current trial was preceded by a frequent sample trial than by an infrequent sample trial (DiGian and Zentall, in press, Experiment 3).

To distinguish between the effects of sample frequency and those of the frequency of comparison response, Zentall and Clement (2002) trained pigeons on two independent matching tasks. In one task, red and green samples were matched to red and green comparison stimuli. In the other task, red and white samples were matched to circle and dot comparison stimuli. Thus, there were four comparison stimuli (red and green hues, and vertical and horizontal lines) but there were only three samples, red, green, and white hues. In the absence of memory for the sample, each comparison stimulus was associated with 50% reinforcement when it appeared (or was associated with reinforcement on about 25% of the trials), however, red samples were presented on 50% of the trials and green and white samples were presented on 25% of the trials each. With this design, sample frequency varied but the probability of a response to each of the comparison stimuli was comparable. And most important, given the presence of one pair of comparisons or the other, the probability that the sample had been red was the same as the probability that it had been green (in one case) or white (in the other case).

Once again, the results showed that when a delay was inserted between the offset of the sample and the onset of the comparison stimuli, divergent retention functions were found. In the absence of memory for the sample, pigeons chose the comparison stimulus associated with the red sample more than the comparison associated with either the green or the white samples. Thus, we confirmed the dissociation of sample frequency from the probability of reinforcement associated with each of the comparison stimuli.

The purpose of the present experiment was to determine the effect of the probability of a comparison response in training on choice of the comparison stimulus when sample frequency and the number of reinforcements associated with each of the comparison stimuli were equated. To accomplish this, pigeons were trained with red, green and yellow samples and red and green comparison stimuli. For one counterbalancing group, when the sample was green, choice of the green comparison was reinforced 100% of the time. However, when the sample was either red or yellow, choice of the red comparison was reinforced but only 50% of the time. Green, red, and yellow samples were each presented an equal number of times. Thus, although there were twice as many trials on which the red comparison stimulus was ‘correct,’ because those responses were reinforced only 50% of the time, the number of reinforcements obtained for comparison choice in training was also equal.

Although there is evidence that a 10-sec intertrial interval is sufficient to eliminate the proactive effects of the sample or the correct comparison from the preceding trial (Zentall and Clement, 2002), because all trials were more likely to be preceded by a trial on which a correct response would have been made to the more frequently chosen comparison, trials on which the more frequently chosen comparison was correct were followed by a 30-sec intertrial interval. To further ensure that proactive interference from the preceding trial on which the more frequently chosen comparison was correct could not account for a more frequently chosen comparison bias, the intertrial interval following a trial on which a response would have been made to the less frequently chosen comparison was shortened to 10 sec. Thus, if there were any proactive effects from the preceding trial it would be expected to result in a bias to choose the less frequently chosen comparison rather than a bias to choose the more frequently chosen comparison.

2. Materials and Methods

2.1. Subjects

The subjects were eight 5-8 year old White Carneaux pigeons (Columba livia) of undetermined sex, purchased from the Palmetto Pigeon Plant (Sumter, SC). The pigeons had previously served in an experiment involving simple simultaneous discriminations. They were maintained at 85% of their free-feeding body weight for the duration of the experiment and were caged individually with free access to grit and water in the home cage. All pigeons were cared for in accordance with University of Kentucky animal care guidelines. The colony room was maintained on a 12:12-h light:dark cycle.

2.2 Apparatus

The experiment was conducted in a BRS/LVE (Laurel, MD) sound-attenuating pigeon test chamber. Three round response keys (2.5 cm × 2.5 cm) were aligned horizontally on the response panel and were separated by 1 cm. A 12-stimulus in-line projector (Industrial Electronics Engineering, Van Nuys, CA) with 28 V, 0.1 A lamps (GE 1820) was mounted behind each response key. The center response key projected red, green, and yellow hues (Kodak Wrattan Filter Nos. 26, 60, and 9, respectively). The left and right response keys projected red and green hues. A houselight located at the center of the chamber ceiling provided general illumination during intertrial intervals (ITI). A rear-mounted grain feeder provided mixed grain reinforcement (Purina Pro Grains) through an aperture centered horizontally on the response panel. Reinforcement consisted of 1.5-s access to mixed grain. An exhaust fan mounted on the outside of the chamber masked extraneous noise. The experiment was controlled and data collected by a microcomputer located in the adjacent room.

2.3. Procedure

2.3.1. Zero-delay conditional discrimination training

Training trials began with the presentation of either a green, red, or yellow sample light on the center key. Ten pecks to this key darkened it and resulted in the immediate presentation of green and red comparisons on the side keys. For half of the pigeons, the red comparison was correct following both red and yellow samples and the green comparison was correct following green sample trials. Correct comparison choices on red and yellow sample trials were reinforced 50% of the time and were followed by a 30-sec ITI. Incorrect choices on these trials resulted in trial termination and a 30-sec ITI only. Correct comparison choices on green sample trials were reinforced 100% of the time followed by a 10-sec ITI. Incorrect choices on green sample trials resulted in trial termination and a 10-sec ITI only (see Figure 1).

Figure 1.

The three trial types used in the experiment. Each sample appeared equally often in each session such that reinforcement associated with each comparison was equated but choice of the two comparisons was biased. For half of the pigeons the sample associated with less frequent choice of the correct comparison was green (shown). For the remaining pigeons the sample associated with less frequent choice of the correct comparison was red (not shown)

For the remaining pigeons, the green comparison was correct following green and yellow sample trials and the red comparison was correct following red sample trials. Correct responses on green and yellow sample trials were reinforced 50% of the time and were followed by 30-sec ITIs. Correct responses on red sample trials were reinforced 100% of the time and were followed by 10-sec ITIs. Incorrect responses resulted in trial termination and the 30-sec (on green and yellow sample trials) or 30-sec (on red sample trials) ITI only.

Each training session consisted of 72 trials, with an equal number of red, green, and yellow sample trials. One comparison choice was correct twice as often as the other, but it was only reinforced 50% of the time. Thus, there were an equal number of potential reinforcements earned for red-comparison-correct trials and green-comparison-correct trials. Training sessions were conducted once a day, 6 days a week. The pigeons remained in training until they reached a criterion of 90% correct or better on each of the trial types for 2 consecutive sessions.

2.3.2. Testing

Testing sessions were similar to training sessions with the exception that there was a delay inserted between sample offset and comparison onset of 0, 2, 4, or 8 sec. There were 18 trials at each retention interval for a total of 72 trials per session. Pigeons remained in testing for 16 sessions. To obtain an accurate estimate of the slope of the retention functions, testing sessions were included only if 0-sec-delay matching accuracy was at least 83% for each of the three trial types. This meant dropping a small number of sessions for most of the pigeons (0-4) and 9 sessions for one pigeon. The performance of one pigeon was quite disrupted when delays were introduced and that pigeon required 37 sessions to reach the 83% matching accuracy criterion at the 0-sec delay. In all analyses the .05 level of significance was adopted.

3. Results

3.1. Acquisition

The pigeons acquired the matching task in an average of 36 sessions (SEM ± 11.9). At the end of training, matching accuracy on the three training trial types was comparable. It was 96.6% correct (SEM ± 1.04), for the two 50% reinforcement trial types and 94.5% correct (SEM ± 0.87) for the 100% reinforcement trial type, a difference that was not statistically significant F(1, 7) = 1.87, p > .05.

3.2. Testing

When a delay was introduced between the offset of the sample and the onset of the comparisons, the pigeons’ matching accuracy declined but it did so faster for trial types involving a correct comparison which was chosen half as often during training. That is, as errors attributable to the delay were introduced, the pigeons showed a bias to choose the more frequently chosen comparison from training. Because matching accuracy for the two trial types associated with the more frequently chosen comparison was comparable, matching performance for those two trial types was combined. The results of delay testing, plotted separately for Test Sessions 1-8 and 9-16, appear in Figure 2.

Figure 2.

Retention functions for trial types involving the comparison that was more frequently chosen in training (Frequent) and the trial type involving the comparison that was less frequently chosen in training (Infrequent), Tests Sessions 1-8 on the left and Test Sessions 9-16 on the right.

A three-way analysis of variance performed on the testing data with delay (0, 2, 4 & 8 sec), trial type (frequently chosen comparison vs. infrequently chosen comparison from training), and test session block (Sessions 1-8 versus Sessions 9-16) as factors confirmed that there were differences in trial-type matching accuracy. Although the pigeons improved their matching accuracy from Test Sessions 1-8 to Test Sessions 9-16 they did not do so differentially for the two trial types.

The analysis indicated that in addition to a significant effect of Delay F(3, 21) = 65.61, p < .001, there was a significant effect of Trial Type F(1, 7) = 26.83, p = .002, and a significant effect of Test Session Block, F(1, 7) = 17.56, p = .004. As expected there was a significant Delay X Trial Type interaction, F(3, 21) = 7.00. p = .002, and there was also a significant Delay X Test Session Block interaction, F(3, 21) = 3.63, p = .03. The latter effect indicated that on later test sessions the pigeons improved more at the longer delays than at the shorter delays. On the other hand, the difference between trial types was similar for the two blocks of test sessions (the Trial Type X Test Session Block interaction), F < 1, and the Delay X Trial Type interaction also was similar for the two blocks of test sessions (the three-way interaction), F < 1.

3.3. Signal Detection Analyses

Although graphs of percentage correct at each delay show the divergent functions indicative of a response bias, Johnstone and Alsop (1999) have suggested that a signal detection analysis, in which log bias (b) is separated from log discriminability (d), provides a better independent assessment of response bias. Davison and Tustin (1978) have proposed that response bias can be calculated with the equation:

$$\log \mathrm{b}=0.5\log \left(\frac{\mathit{Cfr}.\mathit{cor}\times \mathit{Cinf}.\mathit{incor}}{\mathit{Cinf}.\mathit{cor}\times \mathit{Cfr}.\mathit{incor}}\right)$$

and discriminability can be calculated with the equation:

$$\log \mathrm{d}=0.5\log \left(\frac{\mathit{Cfr}.\mathit{cor}\times \mathit{Cinf}.\mathit{cor}}{\mathit{Cfr}.\mathit{incor}\times \mathit{Cinf}.\mathit{incor}}\right)$$

where Cfr.cor is the number of correct responses to the more frequently correct comparison stimulus, Cfr.incor is the number of incorrect responses to the more frequently correct comparison stimulus, Cinf.cor is the number of correct responses to the less frequently correct comparison stimulus, and Cinf.incor is the number of incorrect responses to the less frequently correct comparison stimulus.

Response bias as a function of delay appears in Figure 3. As can be seen in Figure 3, there is a substantial response bias especially at delays greater than 0 sec. A one-way repeated measures analysis of variance performed on the bias data indicated that the bias did not change significantly as a function of delay, F(3, 21) = 1.42, p = .27, but the variability in bias was quite large. Follow-up t-tests performed on the bias scores at each delay indicated that although the bias was not significant at the 0-sec delay, t(7) = 1.55, it was significant at the 1-, 2-, and 4-sec delays, t(7) = 4.87, 3.83, and 3.78, respectively.

Figure 3.

Bias (log b) to choose the comparison that was more frequently chosen in training as a function of delay (see text).

Discriminability as a function of delay appears in Figure 4. As can be seen in Figure 4, discriminability decreases substantially with increasing delay. A one-way repeated measures analysis of variance performed on the discriminability data indicated that the decrease in discriminability was significant, F(3, 21) = 130.68, p < .001. Again, follow-up t-tests performed on the discriminability scores at each delay indicated there was significant discriminability at all delays (0, 2, 4, and 8 sec), t(7) = 15.99, 5.13, 5.15, 4.43, respectively, all ps < .003.

Figure 4.

Discriminability (log d) as a function of delay (see text).

4. Discussion

The results of the present study indicate that response frequency during training can play a role in comparison choice when error-inducing delays are introduced. Thus, in the absence of a difference in sample frequency or probability of reinforcement for choice of comparison (when the sample is not available) pigeons have a bias to choose the comparison that they chose most often during training.

Although there was no programmed difference in reinforcement for comparison choice, the bias shown by the pigeons to choose the comparison most frequently chosen in training did produce an increase in reinforcement and nonreinforcement associated with the bias. However, any differential reinforcement and nonreinforcement did not produce the bias but resulted from the bias. The fact that there was no significant change in the bias from the first half of test sessions to the second empirically supports this interpretation.

The results of the present experiment complement earlier research showing that pigeons’ choice of comparison stimulus is influenced by more than (1) the conditional probability of reinforcement for comparison choice given a sample and (2) the probability of reinforcement for comparison choice in the absence of a sample, as others have proposed. DiGian and Zentall (in press) found that in the absence of memory for the sample, a combination of greater sample frequency and a higher proportion of responses to one of the two comparisons in training can influence comparison choice. And Zentall and Clement (2002) showed that when delays are introduced, sample frequency alone, in the absence of differential reinforcement for comparison choice or more frequent responding to one of the comparison stimuli during training, can influence comparison choice. The results of the present study show that in the absence of differential reinforcement for comparison choice and in the absence of differential experience with the sample stimuli, when delays are introduced, the proportion of responses to the two comparisons experienced during training is sufficient to bias the pigeons to choose the comparison to which they responded more frequently in training.

This series of experiments has implications for a general theory of reinforcement. It suggests that comparison choice is affected by more than the probability of reinforcement given a response. It implies that there is a discontinuity between choice and reinforcement such that overall sample frequency and overall probability of comparison choice in training are taken into consideration when making comparison choice on test trials. In addition to considering the probability of reinforcement associated with a choice, it appears that pigeons ‘consult’ their long-term store of sample frequencies and comparison choices in choosing between comparison stimuli on test trials when the sample is not available.