Contingency Tracking During Unsignaled Delayed Reinforcement

Abstract

Three experiments were conducted with rats in which responses on one lever (labeled the functional lever) produced reinforcers after an unsignaled delay period that reset with each response during the delay. Responses on a second, nonfunctional, lever did not initiate delays, but, in the first and third experiments, such responses during the last 10 s of a delay did postpone food delivery another 10 s. In the first experiment, the location of the two levers was reversed several times. Responding generally was higher on the functional lever, though the magnitude of the difference diminished with successive reversals. In the second experiment, once a delay was initiated by a response on the functional lever, in different conditions responses on the nonfunctional lever either had no effect or postponed food delivery by 30 s. The latter contingency typically lowered response rates on the nonfunctional lever. In the first two experiments, both the functional and nonfunctional levers were identical except for their location; in the third experiment, initially, a vertically mounted, pole-push lever defined the functional response and a horizontally mounted lever defined the nonfunctional response. Higher response rates occurred on the functional lever. These results taken together suggest that responding generally tracked the response–reinforcer contingency. The results further show how nonfunctional-operanda responses are controlled by a prior history of direct reinforcement of such responses, by the temporal delay between such responses and food delivery, and as simple generalization between the two operanda.

New responses are learned in the absence of temporal contiguity between them and the reinforcers they produce (Byrne, Lesage, & Poling, 1997; Critchfield & Lattal, 1993; Lattal & Gleeson, 1990; LeSage, Byrne, & Poling, 1996; Wilkenfield, Nickel, Blakely, & Poling, 1992; Williams & Lattal, 1999). These responses are easily developed with 30-s delays and have been reported with delays of reinforcement of up to 60 s (Avila & Bruner, 1995); however, a claim that the responses are controlled by the temporally extended response–reinforcer relation requires that other potential sources of control over the response be excluded. Evidence in support of the claim comes from the findings that responses are neither established nor maintained in the absence of reinforcement nor when otherwise identical reinforcers are delivered independently of the responses (Gleeson & Lattal, 1987; Lattal & Gleeson, 1990).

A further test of the limits of the differential sensitivity of responding to response–reinforcer relations involves the use of two operanda. Responding on one of them, hereafter labeled the functional lever, produces food after an unsignaled delay period as noted above. Responses on the other, hereafter labeled the nonfunctional lever, are recorded, but without other programmed effect. The distribution of responses between the two operanda ideally would index the extent to which responding tracks, that is, is sensitive to, the response–reinforcer relation, or, as it often is described, the contingency (cf. Williams & Lattal, 1999). The present experiments examined such tracking to further assess the control of operant responding by delayed consequences.

The results of two previous experiments bear on contingency tracking. Wilkenfield et al. (1992) reported higher response rates of rats on a functional lever than on a nonfunctional one when each functional response initiated unsignaled delays of 8 s or less. When Wilkenfield et al. used 16- or 32-s resetting delays to reinforcement, however, responding on the nonfunctional lever was more frequent than on the functional lever. By contrast to the latter finding, Critchfield and Lattal (1993) reported little or no responding by rats on a nonfunctional lever during the acquisition of a spatially defined operant with delayed reinforcement. Specifically, each break of a photocell beam across the rear of the experimental chamber initiated an unsignaled 30-s resetting delay that terminated with a food pellet delivery.

Wilkenfield et al.'s (1992) findings are theoretically important because they raise questions about observations (e.g., Avila & Bruner, 1995, Gleeson & Lattal, 1987; Lattal & Gleeson, 1990; Richards, 1981), noted above, that response acquisition and maintenance in experimentally naive and otherwise untrained animals are possible with relatively long temporal gaps between the reinforcer and the response that produces it. Specifically, their data could be taken to suggest that the response–reinforcer relation is effective in controlling responding only when reinforcers occur within about 8 s of a response. A different interpretation of the Wilkenfield et al. findings, however, is that nonfunctional-lever responding is maintained by variables independently of, or interdependently with, the response-delayed reinforcer relation in effect on the functional lever. At least three such variables might contribute to high rates of nonfunctional-lever responding. First, in the Critchfield and Lattal (1993) experiment, topographically different responses served as the functional and nonfunctional responses, whereas Wilkenfield et al. used identical levers for either response. Thus, nonfunctional-lever responding might occur as a result of simple generalization from one operandum to the other. Second, responses on the nonfunctional lever may occur in closer proximity to food delivery than those on the functional lever and, therefore, may be maintained by less-delayed, but unprogrammed, reinforcement. Wilkenfield et al., for example, did not include any contingency to ensure temporal separation between responses on the nonfunctional lever and delivery of food for responding on the functional lever.

A third variable, not addressed by Wilkenfield et al. (1992), that may affect the control of nonfunctional-operandum responding is the prior history of reinforcement correlated with such responding. Williams and Lattal (1999), for example, reinforced pigeons' functional-key pecks according to a tandem variable-interval (VI) 15-s differential-reinforcement-of-other-behavior (DRO) 10-s schedule of reinforcement such that an unsignaled transition from the VI component to the DRO component occurred on the first response after the VI interval had lapsed. Pecks on a second, nonfunctional, key had no consequence unless they occurred during a delay initiated by a functional-key peck, in which case the upcoming reinforcer was delayed 10 s. Williams and Lattal reversed the position of the functional response every one, two, or three sessions. Because of the frequent alternation of the functional and nonfunctional keys, responding on the latter persisted at relatively high rates, but the ratio of functional key pecks to total key pecks increased the longer the functional response was fixed at the same location.

As already discussed, contingency tracking bears on an understanding of the temporal relations involved in reinforcement. The present experiments therefore were conducted to examine, first, the development of the tracking of a response-delayed reinforcer dependency or its absence. Second, the effects of prior histories of reinforcement, possible unprogrammed reinforcement of nonfunctional-operandum responses, and the physical similarity between the functional and nonfunctional operanda were examined to further isolate sources of control over the functional responses.

Experiment 1

This experiment examined the extent to which operant responding tracks the contingencies as the location of the lever correlated with an unsignaled, resetting delay of reinforcement procedure was reversed successively. As noted above, Williams and Lattal (1999) found increased tracking accuracy with increasing numbers of sessions in which the functional lever remained constant in one location. In the present experiment, the locations of functional and nonfunctional operanda remained fixed over repeated sessions, with a 30-s resetting delay operative on the functional lever.

Method

Subjects

Each of 3 female Sprague-Dawley rats, approximately 120 days old at the start of the experiment, was maintained at 75% of its ad libitum body weight by postsession feeding. Each was housed individually and had unrestricted access to water except during experimental sessions.

Apparatus

Two Ralph Gerbrands Co. Model G7010 rat conditioning chambers were enclosed in sound-attenuating chambers. The chambers were 20.5 cm wide by 19.5 cm high by 23.5 cm long. The aluminum work panel contained two horizontal rat levers, 5 cm long by 1.2 cm wide, operated by a force of 0.25 N. The levers were located 8.0 cm from the floor of the chamber and 6.0 cm from the left and right edges of the panel. The panel also contained on its midline a 4.4 cm square food aperture into which 45-mg food pellets could be delivered by a Gerbrands Model G5100 pellet dispenser. The bottom edge of the food aperture was located 0.75 cm from the floor. General illumination was provided by a white houselight (3 W, 28-V DC) located on the horizontal midpoint of the work panel 11 cm from the floor. White noise and a ventilation fan on the chamber enclosure masked extraneous noise. An IBM-compatible microcomputer using a Med-PC® interface and software was located in an adjacent room, and it controlled all experimental events and recorded data.

Procedure

Magazine training began when each rat reached its target weight. A food pellet was delivered immediately after the rat was placed in the experimental chamber and the houselight turned on. Pellets were delivered thereafter approximately every 15 s after the previous one had been consumed. When the latency from pellet delivery to consumption decreased to less than 1 s, as assessed by direct observation of the rat, pellets then were delivered according to a variable-time (VT) 30-s schedule. The levers were present during this magazine training but responses on them were without effect. Magazine training ended, and the first experimental session began, when the latency from pellet delivery to consumption was 2 s or less for 10 consecutive pellet deliveries (also assessed by direct observation). Magazine training sessions lasted no more than 30 min per session and required one or two sessions.

During each session of the first condition, responding on the right lever was reinforced according to a tandem fixed-ratio (FR) 1 DRO 30-s schedule of reinforcement, that is, each response initiated a 30-s unsignaled, resetting delay-to-reinforcement interval. This procedure will be described hereafter as an unsignaled delay. The lever correlated with the unsignaled delay was designated the functional lever. Responding on a second lever, located on the left side of the work panel during the first condition, did not initiate delays leading to reinforcement. If, however, a response occurred on this nonfunctional lever during the last 10 s of a delay initiated by a functional-lever response, the upcoming pellet delivery was postponed by 10 s. Subsequent responses on the nonfunctional lever during the delay period restarted this 10-s interval.

The first condition continued until response rates on the functional lever were consistently higher over at least 10 sessions than response rates on the nonfunctional lever. The positions of the functional and nonfunctional levers then were reversed. In subsequent conditions, the positions were reversed twice more for Rats JK2 and JK3 and once more for Rat JK4. The number of sessions that each condition was in effect for each rat is shown in Table 1. Sessions occurred 5 days a week at the same time and lasted for 2 hr or until 60 reinforcers were delivered, whichever occurred first. All 60 reinforcers were obtained within the allotted time in most sessions.

Table 1.

Number of sessions in each condition during Experiment 1.

	Functional Lever Position
	Condition
	1	2	3	4
Rat	Right	Left	Right	Left
JK2	89	34	58	53
JK3	116	40	32	47
JK4	81	97	15

Results

The leftmost panel of Figure 1 shows that responding on the functional and nonfunctional levers was differentiated for each of the 3 rats after 2 to approximately 40 sessions, with higher response rates on the functional lever. After varying numbers of sessions following the first and second reversal for Rat JK2 and following the first, second, and third reversal for Rat JK3, response rates reversed, with consistently higher rates occurring on the functional lever toward the end of these respective conditions. Rat JK4 failed to show reversals in response rates during either reversal, as did Rat JK2 during the third reversal. Each of these reversal failures reflected continued relatively high response rates on the nonfunctional lever.

Fig 1.

Responses per min on the functional and nonfunctional levers across conditions of Experiment 1. “Right” and “Left” describe the location of the functional lever in the conditions shown in the graphs.

Figure 2 shows discrimination ratios (the ratio of the number of functional-lever responses to total responses) across reversals. Ratios above 0.5 indicate more responses on the functional lever. The discrimination ratios show that in each condition for Rats JK2 and JK3, the ratios increased across successive sessions following the reversal, eventually rising and remaining above .5 in all but the last reversal for Rat JK2. In line with the data in Figure 1, the discrimination ratios for Rat JK4 mostly remained below 0.5 following both reversals and despite extended training during the first reversal. During both the first and second reversals with this rat, the discrimination ratios did increase across the first few sessions following the reversal.

Fig 2.

Discrimination ratios (functional-lever presses/total lever presses) across conditions of Experiment 1.

The number of reinforcers in each session (out of 60) that were initiated by a nonfunctional-lever press is shown in Figure 3. These reinforcers were those delivered 10 s after a nonfunctional-lever press. Thus, the sequence resulting in this circumstance would be as follows: A functional-lever response initiated the delay interval. Each response on the functional lever reset the interval to 30 s, but responses on the nonfunctional lever were without effect unless they occurred during the last 10 s of the delay. Any nonfunctional-lever response occurring during this period initiated a 10-s delay, at the end of which the reinforcers depicted in Figure 3 were delivered. During the first reversal, Rat JK4 was far more likely to have received a food pellet 10 s from the last response on the nonfunctional lever than were the other 2 rats in this condition. During the third and fourth reversals for Rats JK2 and JK3, at least one or two pellets occurred 10 s after a response on the nonfunctional lever in most sessions.

Fig 3.

Total number of reinforcers delivered 10 s after a nonfunctional-lever press across conditions of Experiment 1.

Discussion

The results from the initial condition of this experiment replicate and extend the findings of Critchfield and Lattal (1993) and Williams and Lattal (1999) in that responding was established and maintained at higher rates on the functional than the nonfunctional lever. These results do not replicate the findings of Wilkenfield et al. (1992) because they found higher response rates on the nonfunctional lever at 32 s delays to reinforcement programmed similarly to the delays in the present experiment. Two differences between Wilkenfield et al.'s procedure and the present experiment may have contributed to the different results. First, Wilkenfield et al.'s experiment lasted only a single 8-hr session. The data in Figures 1 and 2 show that, during the first condition, the development of consistently higher response rates on the functional lever required, in some cases at least, longer than 8 hr of exposure to the contingencies. Second, Wilkenfield et al. did not arrange for responses on the nonfunctional lever to postpone food delivery, but the present procedure did. This second difference was the topic of the next experiment.

The results of the reversals suggest that as a history of reinforced responding—albeit reinforced 30 s after the response—accumulates, responding on the now-nonfunctional lever becomes more resistant to change. The increased responding on the nonfunctional lever sets into play a complex contingency: Responses on the functional lever produce reinforcers reliably, but each time after a 30-s period of no further responding on the functional lever. No programmed relation existed between nonfunctional responses and 30-s delay initiation, but nonfunctional responses could occur only 10 s from food delivery. Figure 3 shows that the number of reinforcers occurring 10 s after a nonfunctional-lever response typically was low, but was highest for Rat JK4, where responding failed to reverse. Lattal (1974) showed that only a few food deliveries need be contiguous with the response to sustain responding above the level maintained by all response-independent food deliveries. Interspersing a small number of relatively less delayed reinforcers in a stream of otherwise longer-delayed reinforcers may have similar effects. Other factors that likely influenced responding on the nonfunctional lever were simply the history of reinforcement of responding on that lever in preceding conditions, the similarity of the two levers to one another, and, in the case of Rat JK4, perhaps position bias as well.

Experiment 2

As noted in the Discussion of Experiment 1, a major difference between that experiment and Wilkenfield et al. (1992) was the presence of a delay in the former between any responses on the nonfunctional lever and food delivery. Experiment 2 was conducted to directly compare nonfunctional-lever responding in the presence and absence of such a delay.

Method

Subjects

Three experimentally naive female Sprague-Dawley rats were used. The conditions of their maintenance were as described in the first experiment.

Apparatus

A BRS-Foringer operant conditioning chamber for rats was used. The chamber measured 28 cm long by 20 cm wide by 17 cm high and was placed in a sound-attenuating enclosure. Two walls were transparent plastic and the other two were aluminum. One of the aluminum walls served as the work panel. This panel housed two 5 cm long × 1.2 cm wide response levers mounted 4.5 cm from the chamber floor, 15.5 cm apart and equidistant from the side walls. Operation of each lever required a minimum force of 0.25 N. A Gerbrands Model D-1 feeder delivered standard 45-mg Noyes pellets into a pellet tray that protruded 4.5 cm into the chamber from the work panel. The tray was centered 4.5 cm from the chamber floor and midway between the two levers. During each session, the chamber was continuously illuminated by a 3-W 28-V DC white houselight centered on the chamber ceiling. White noise and a ventilating fan mounted on the enclosure masked extraneous sounds. In an adjacent room, an IBM-compatible microcomputer operating MedPC® software programmed experimental events and recorded data.

Procedure

Magazine training was as described in Experiment 1. The first experimental session began immediately after the 10^th consecutive latency of less than 2 s to eat following a pellet delivery to the food aperture. As in Experiment 1, responding on the functional lever was reinforced according to a tandem FR 1 DRO 30-s schedule of reinforcement throughout the experiment. Nonfunctional-lever responses that occurred before a functional-lever response- initiated delay were recorded but were without other effect. In different conditions, each response on the nonfunctional lever that occurred during a functional-lever response-initiated delay either did not postpone the reinforcer (0-s delay) or postponed it by 30 s (30-s delay).

Table 2 shows the location of the nonfunctional lever, the delay in effect on that lever, and the number of sessions in each condition for each rat. Conditions were changed when there was neither a positive nor negative trend in response rates on the functional lever over the last six sessions. Sessions occurred 5 days a week and lasted for 2 hr or until 60 reinforcers had been delivered, whichever occurred first. As in Experiment 1, 60 reinforcers usually were obtained within the allotted time.

Table 2.

Order and number of sessions in each condition during Experiment 2.

	Nonfunctional Lever Position and Delay (s)
	Condition
	1	2	3	4	5	6	7
Subject	Right 0 s	Left 0 s	Left 30 s	Left 0 s	Right 0 s	Right 30 s	Right 0 s
TF1	34	24	24	24	20	22	24
	Right 0 s	Left 0 s	Left 30 s	Left 0 s	Left 30 s	Left 0 s
TF2	34	30	23	38	31	53
	Left 0 s	Right 0 s	Right 30 s	Right 0 s	Left 0 s	Left 30 s
TF4	34	45	25	22	50	55

The first condition for each rat scheduled the 0-s delay on the nonfunctional lever, with its location differing across rats as shown in Table 2. Because response rates on the nonfunctional lever were low in this first condition, the locations of the functional and nonfunctional levers were reversed in the second condition in an attempt to engender reliable responding on the nonfunctional lever that subsequently might be suppressed by the delay contingency. This second condition thus served as the baseline against which the effects of adding the 30-s delay following each response on the nonfunctional lever in the third condition were assessed. Then, the 30-s delay was removed from the nonfunctional lever in the fourth condition. The sequence in Conditions 3 and 4 then was repeated with Rat TF2. It also was repeated with Rats TF1 and TF4 after the functional and nonfunctional levers first were reversed in Condition 5; however, because of time constraints, Rat TF4 was not returned to the 0-s delay condition on the nonfunctional lever.

Results

Figure 4 shows response rates on each lever during each session of the experiment. Because the response rates were, in most cases, low, the y-axis has been truncated such that response rates of more than 10 responses per min appear off the scale. The relative rates on the functional lever are represented as discrimination ratios (functional lever presses/total lever presses) in Figure 5, where values above 0.5 indicate more responding on the functional lever.

Fig 4.

Responses per min on the functional and nonfunctional levers across conditions of Experiment 2. “R” and “L” identify the nonfunctional lever as being on the right and left, respectively.

Fig 5.

Discrimination ratios (functional-lever presses/total lever presses) across conditions of Experiment 2.

Responding on both the functional and nonfunctional levers occurred reliably during the first condition, albeit at low rates. By the end of the first condition, response rates on the functional lever were higher than those on the nonfunctional lever for Rats TF1 and TF2. During most of the first condition, response rates of Rat TF4 also often were higher on the functional lever, but, towards the end of that condition, there were several sessions where response rates were higher on the nonfunctional lever. The discrimination ratios in Figure 5 mirror the response rates on the two levers. Rat TF1 showed a gradually increasing discrimination ratio across the condition, while Rats TF2 and TF4 showed no systematic change across the condition. The ratios were rather variable across sessions, but most were above 0.5.

When the location of the functional lever was reversed in the second condition, the effects were mixed. Figure 4 shows that response rates for Rats TF 1 and TF 2 on the nonfunctional lever initially were high, carrying over from the preceding condition where it was the functional lever. The nonfunctional-lever rates during the second condition decreased for Rats TF1 and TF4 over the condition. These rate changes are reflected in Figure 5 as discrimination ratios above 0.5. For Rat TF2, the response rates on the nonfunctional lever initially began to decrease slightly, but then reversed direction and increased toward the end of the condition. Response rates on the functional lever gradually increased across the condition, albeit not enough to exceed the rates on the nonfunctional lever. These rate changes are reflected in the discrimination ratios in Figure 5 as an increasing trend in the first part of the condition followed by a decreasing trend in the second part.

In the next two conditions, the location of the nonfunctional lever remained fixed. The second condition served as a baseline, and these next two conditions allowed assessment of the effects of adding and removing the 30-s delay on the nonfunctional lever, respectively. During the third condition, where the latter delay was in effect, response rates of each rat on the nonfunctional lever decreased relative to the last few sessions of the preceding 0-s condition (see Figure 4). This change also is reflected in Figure 5 as generally higher discrimination ratios across this condition relative to the preceding one. Subsequently, removing the 30-s delay on the nonfunctional lever in the fourth condition changed neither response rates nor discrimination ratios for Rat TF1. This same effect occurred for a few sessions following the removal of the delay with Rat TF2; however, as that condition proceeded its nonfunctional-lever response rates increased substantially and, with those increases, its discrimination ratios fell accordingly (see Figure 5). For Rat TF4, when the 30-s delay was added in the third condition response rates on the functional lever increased slightly relative to the last few sessions of the preceding condition, but removing it in the next condition had no systematic effect on responses rates on either lever relative to the preceding condition.

The fifth, sixth, and seventh conditions for Rat TF1 were a systematic replication of the previous 0-s–30-s–0-s delay conditions but with the right, rather than the left, lever being nonfunctional. Reversing the location of functional and nonfunctional levers for Rat TF1 in the fifth condition resulted in high rates of nonfunctional-lever responding and lower rates on the functional lever. Adding a 30-s delay on the nonfunctional lever in Condition 6, however, reduced nonfunctional-lever response rates below those on the functional lever. Nonfunctional-lever response rate was often zero when the delay was removed in the last condition.

For Rat TF2, the fourth, fifth, and sixth conditions constituted a direct replication of the 0-s–30-s–0-s conditions studied previously, except that the left lever was always the nonfunctional one. The high response rates on the nonfunctional lever observed under the 0-s delay of the fourth condition (as previously noted) were reduced by the 30-s delay contingency. During the sixth condition, removal of the 30-s delay resulted in higher responding on the nonfunctional lever than on the functional lever. The discrimination ratios in Figure 5 reflect these response rate changes.

For Rat TF4, the fifth and sixth conditions were systematic replications of its second and third conditions, except that the nonfunctional lever was on the left. Like Rat TF2, this rat responded with higher rates on the nonfunctional lever following the reversal in lever functions during the fifth condition. In the sixth condition, response rates remained higher on the now-nonfunctional left lever than they were on the now-functional right lever. The discrimination ratios did not reverse in this last condition (see Figure 5).

The data in Figure 6 are, for the last six sessions of each condition, the percentages of all reinforcers that were delivered when the last response before reinforcement was a nonfunctional-lever press. These data show that the likelihood of a reinforcer being delivered after a functional lever response usually was lower when the 30-s delay was in effect on the nonfunctional lever. The exception was Rat TF4.

Fig 6.

Percentage of reinforcers following a nonfunctional response. Data are from the last six sessions of Conditions 2, 3, 4, and 5, except for Rat TF1 where data are from Conditions 2, 3, 4, 5, and 6.

Discussion

In five of six instances, response rates on the nonfunctional lever decreased when a 30-s delay followed responses on the nonfunctional lever during the delay period relative to the rates in the preceding 0-s conditions (the exception was Rat TF4 during its last condition). These findings are consistent with those of the first experiment and support the suggestion that improved temporal contiguity between nonfunctional lever responses and food delivery contributes to the maintenance of these responses. They suggest that the absence of such a delay in the Wilkenfield et al. (1992) experiment contributed to the relatively high rate of responding they observed; however, the result from Experiment 2, like those from the first experiment, also illustrate the complexity of isolating such effects against a background of reversing the location of the functional lever.

In three cases (Rat TF1 in the fourth and seventh conditions and Rat TF4 in the fourth condition), response rates on the nonfunctional lever did not increase following removal of the 30-s delay. In the other cases, the effects of the delay on the nonfunctional lever were reversible. Often, exposing an organism to a particular contingency puts into place patterns of behavior that persist following removal of the contingency that established them, most likely because the existing response patterns do not interfere with obtaining reinforcement. For example, Marcucella and MacDonall (1977) observed that high response rates persisted once a positive behavioral contrast-inducing contingency was removed. This occurred because responding was maintained on a VI schedule, wherein high response rates did not interfere with reinforcement and may have been reinforced adventitiously. Similarly, in those present instances where nonfunctional-lever response rates did not increase following removal of the 30-s delay, responding on the nonfunctional lever did not add to the reinforcement rate after having decreased it in the previous condition. That is, the low rates of responding on the nonfunctional lever persisted.

The main results of Experiment 2 complement earlier findings of Sutphin, Byrne, and Poling (1998). In their experiment, each response by rats on a nonfunctional lever during a delay period that was initiated by a response on a functional lever canceled the forthcoming reinforcer. Compared to a group of rats exposed to the same procedure, but without the cancellation contingency, the latter contingency reduced the number of nonfunctional-lever responses. The present results demonstrate that reinforcement need not be canceled but only delayed by a response to produce such an effect. Furthermore, here these effects were shown within individual subjects rather than across groups of subjects.

In both Sutphin et al. (1998) and the present experiment, responses on the nonfunctional lever continued to occur relatively frequently despite the contingencies designed to reduce such responding. In the present experiment, responding was delayed equally from reinforcement on both levers when the 30-s delay was in effect on the nonfunctional lever. Such nonfunctional-lever responding was less reliably correlated with food delivery because there were occasions where a reinforcer followed the absence of responding on that lever, as opposed to the fact that reinforcers always followed (albeit it after the delay) responses on the functional lever. Thus, responding on the nonfunctional lever might be expected to occur at lower rates but also be sustained because it likely occasionally was related adventitiously to food delivery. In Sutphin et al.'s experiment, responding on the nonfunctional lever was never followed by the reinforcer. The fact that it continued under such circumstances suggests that still other variables may have contributed to its maintenance. One of these was the topic of the final experiment.

Experiment 3

Unlike Wilkenfield et al. (1992), Critchfield and Lattal (1993) reported little or no responding to a nonfunctional lever during acquisition of a spatially defined operant. In the latter experiment, photocell breaks by rats were reinforced according to a tandem FR 1 DRO 30-s schedule of reinforcement, and lever presses had no effect. This finding contrasts with both Wilkenfield et al.'s findings and the present findings that the shorter the programmed delay from a nonfunctional response to a reinforcer delivery, the greater the rate of nonfunctional responding. The first condition of the present Experiment 2 was a replication of Critchfield and Lattal's Experiment 1 procedure except that the functional and nonfunctional responses were both lever presses. That lever pressing in Experiment 2 occurred at a higher rate than it did in Critchfield and Lattal suggests that topographical similarity of the functional and nonfunctional operanda may contribute to nonfunctional-lever responding. Experiment 3 was conducted to evaluate further the possible contribution of generalization to nonfunctional-lever responding.

Method

Subjects

Three experimentally naive female Sprague-Dawley rats were used. The conditions of their maintenance were as described in the first experiment.

Apparatus

The chamber used was the same as in Experiment 1 except that the left horizontal lever was removed and an omnidirectional vertical lever (Ralph Gerbrands Company Model G6313) was inserted at the center point of the ceiling of the chamber, where it protruded downward 10.7 cm. A force of 0.20 N exerted in any direction activated the switch attached to the lever and recorded a response. Other details of the apparatus were as described in Experiment 1.

Procedure

Magazine training proceeded as described in Experiment 1. The first session began, as in Experiment 1, immediately after the 10^th latency of less than 2 s to eat following a pellet delivery to the food aperture. In the first condition of the experiment, the functional and nonfunctional levers were, respectively, the vertical and horizontal levers. Responding on the functional lever was reinforced according to a tandem FR 1 DRO 30-s schedule of reinforcement throughout the experiment. Responses on the nonfunctional lever had no consequences unless they occurred in the last 10 s of a delay initiated by a response to the functional lever. Each such response postponed pellet delivery by 10 s.

The first condition remained in effect until responding was reliably differentiated between the two levers. The functional and nonfunctional levers then were reversed such that the horizontal lever became the functional lever and the vertical lever became the nonfunctional lever. Each condition remained in effect until response rates on the two operanda were nonoverlapping for several sessions. The number of sessions that each condition was in effect for each rat is shown in Figure 7. Sessions occurred 5 days a week and lasted for 2 hr or until 60 reinforcers had been delivered, whichever occurred first. Sessions only occasionally lasted for 2 hr.

Fig 7.

Responses per min on the functional and nonfunctional levers across conditions of Experiment 3.

Results

Figure 7 shows responses per min for each subject during each session of the experiment. Acquisition of vertical lever responding occurred after one to eight sessions. As in the other experiments, several sessions were required before functional-lever responding occurred more frequently than did responding on the nonfunctional lever. Response rates on the nonfunctional lever did not exceed one response per min during the first condition. For Rats 1 and 2, response rates on the functional lever were more than six times that on the nonfunctional lever at the end of the first condition. Reversing the functions of the two levers reversed the response rates of all 3 rats on the two levers, although the effect took a longer time to develop with Rat 3 and, when it did, response rates on both levers were low in comparison to those of the other 2 rats.

Figure 8 shows the discrimination ratios for each rat during each condition of the experiment. The ratios for Rats 1 and 2 approached 1.0 (exclusive responding to the vertical lever) at the end of the first condition. Rat 3 had ratios of approximately 0.8 at this point. Similarly high discrimination ratios were obtained at the end of the second condition as well, where the functions of the two levers were reversed.

Fig 8.

Discrimination ratios (functional-lever presses/total lever presses) across conditions of Experiment 3.

Discussion

Using two types of levers resulted in high levels of tracking the response-reinforcer contingency. This effect was replicated when the functions of the levers were reversed, suggesting that such effects were not specific to one type of functional lever. The different levers themselves, their greater physical separation relative to that in the first two experiments, and the changes in movements required by the location and type of lever all might contribute to the high levels of tracking. The closing of the laboratory prior to an extended move to a new facility precluded a direct, within-subject comparison of the effects of nonfunctional levers that were the same or different from one another. A preliminary comparison may be made, however, between these results and those of Experiment 1, where the contingencies were the same, except that, in the latter experiment, two identical levers located to the left and right of the food hopper served as the operanda. Comparing the results of Experiment 3 to the first two conditions of Experiment 1 reveals that, although functional-lever response rates are variable both within and between subjects, in relative terms, nonfunctional-lever response rates were consistently higher in Experiment 1. This result was repeated when the lever functions were reversed in the two experiments during the second condition of each.

The differences in responding between Experiments 1 and 3 suggest that the topography and location of the response at least partially determines nonfunctional-lever responding. It follows that the more similar the operanda, the more likely responding is to occur to both. This may account for why Critchfield and Lattal (1993) observed almost no nonfunctional-lever responding, but Wilkenfield et al. (1992) and Sutphin et al. (1998) did.

General Discussion

Tracking of the response–reinforcer relation, as reflected in a discrimination index, in a two-lever environment where responding on only one of the operanda is reinforced is highly likely when reinforcement is immediate. Such a high discrimination index would reflect optimal control of behavior by the response–reinforcer relation. Wilkenfield et al. (1992) suggested that tracking breaks down at response–reinforcer delays of around 8 s, possibly reflecting diminished control by the response–reinforcer relation. As longer delays are imposed between the response and its reinforcer, however, not only is direct control of responding by the degraded response–reinforcer relation diminished (e.g., Richards, 1981; Sizemore & Lattal, 1977), but the present experiments suggest that the operation of other variables that might enhance responding on the nonfunctional lever are more likely to come into play. Tracking the response–reinforcer relation can occur reliably and consistently with delays of up to 30 s, but variables other than those related to direct control by the response–reinforcer relation may operate to mask or interfere with control of responding during the response–reinforcer delay period. Such variables, either individually or in concert with one another, account for the increased responding on the nonfunctional lever. These variables include a prior history of reinforcement of responding on the (presently) nonfunctional lever, the temporal relations that develop between nonfunctional-lever responding and food delivery, and the distinctiveness of the functional and nonfunctional levers.

Reversing the conditions of reinforcement for responses to the two operanda, perhaps several times, is necessary to assess how well responding tracks the response–reinforcer contingency. But, as the results of Experiment 1 and Williams and Lattal (1999) show, a history of reversed functions on both operanda makes responding on the nonfunctional operandum more resistant to change, at least in part because responding on both operanda is intermittently reinforced across conditions. The degraded response–reinforcer dependency and the DRO requirement (in effect to ensure equivalence of the nominal and obtained delays) add to the complexity because, with such contingencies, only low rates of responding on the functional operandum can occur if reinforcers are to be earned.

The results of the present experiments also suggest some limits to employing the distribution of responses between the two levers as an index of contingency tracking or response sensitivity to the response–reinforcer dependency. The most significant of these is that the two-lever procedure introduces more complex contingencies than may be apparent on initial consideration. Scheduling reinforcers following responses on one operandum necessarily introduces the possibility of an adventitious relation between responding on the nonfunctional operandum and reinforcement. As sometimes happened in the first experiment, and depending on the delay values in effect for responding on the nonfunctional lever, reinforcers can occur more closely in time to responses on the nonfunctional lever. The result may be higher rates of responding on that lever not because of failures to track per se, but because of the adventitious relation between responding on that lever and food delivery.

The results of Experiment 3 show that using physically different functional and nonfunctional operanda allows more accurate tracking of the response–reinforcer dependency in part by decreasing generalization across the two operanda, that is, by making the two devices more discriminable. Such generalization may be stimulus, response, or both. This method does not necessarily assure accurate tracking, however, because biases or preferences for one or the other operandum may exist or develop as a function of many variables such as anatomical features of the organism or differences in effort required by the two operanda.

As the response–reinforcer relation is extended in time, response rates decrease to low levels (Wilkenfield et al., 1992). Because reinforcer delivery per se sometimes may elicit or evoke low rates of lever pressing (e.g., Segal, 1959) on the order of those seen in the present and related experiments, the two-lever procedure has been proposed as a means of isolating contingency tracking or contingency sensitivity from food-elicited responding. The data in each of the experiments suggest the operation of other variables that might contribute to such sustained responding on the nonfunctional lever, independently of, or in conjunction with, food-elicited responding. The appeal of the two-operandum procedure to isolate food-elicited responding from operant responding is that it potentially can reveal these effects concurrently with an assessment of the effectiveness of the contingency. Moreover, the assessment can be done within individual subjects. Lattal and Gleeson (1990) assessed elicitation effects by using an independent group of subjects, each of which received response-independent food deliveries yoked such that they occurred at the same rate and with the same temporal distribution as did reinforcers earned by a group wherein responses were reinforced after unsignaled delays. Critchfield and Lattal (1993) used a within-subjects procedure whereby a no-reinforcement baseline of several sessions preceded the introduction of the delayed reinforcement procedure. Using either VT or extinction as the baseline condition, however, may bias the results against obtaining acquisition because of the development of competing behavior by the baseline contingencies. The present findings suggest that the two-lever procedure is of limited utility in assessing food-, or, more generally, reinforcer-induced behavior because of the concurrent operation of other behavioral processes that may contribute to responding on the nonfunctional operandum.

Although the two-operandum procedure as a means of assessing contingency tracking is subject to the limitations reported herein, there remains substantial evidence that such tracking occurs over delays to reinforcement of at least 30 s, taking into account both “spontaneous” lever pressing and food-elicited responding (Critchfield & Lattal, 1993; Lattal & Gleeson, 1990). The data from the present experiments add further support to the previous analyses of contingency tracking and begin to elucidate the controlling variables. It should be noted, too, that our use of the term “contingency tracking” is as a global description of the behavior of the animal under certain contingencies of reinforcement. As with any reinforcement contingency, behavior is a function of both sensitivity or discriminability variables and direct reinforcement effects (cf. Lattal, 1979). This may be a fruitful question for further experiments to address with respect to establishing the temporal limits of reinforcement effects.