Conditioned Reinforcement Value and Resistance to Change

Abstract

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus–reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.

Resistance to change of operant behavior typically is examined using multiple schedules of reinforcement arranging differential conditions of reinforcement in the presence of two component stimuli. Once a baseline is established, a disruptor (e.g., extinction or satiation) is introduced and resistance to change is measured by examining the resulting decrease in response rates relative to predisruption baseline response rates. Relatively smaller decreases from baseline reflect behavior that is more resistant to change. The typical finding is that responding maintained by higher rates of reinforcement is more resistant to disruption than is responding maintained by lower rates of reinforcement (see Nevin, 1992, for review).

Behavioral momentum theory provides a framework for understanding how differential reinforcement conditions impact response strength. The theory suggests that response rates and resistance to change are two separable aspects of operant behavior (see Nevin & Grace, 2000, for review). Response rates are governed by the contingent relation between responses and reinforcers (i.e., the response–reinforcer relation), but resistance to change is governed by the Pavlovian relation between the discriminative-stimulus context in which the behavior occurs and the reinforcers obtained in that context (i.e., the stimulus–reinforcer relation). From the perspective of behavioral momentum theory, resistance to change provides a measure of response strength that is superior to simple response rates because response rates are influenced by operations that may or may not impact strength. For example, different types of schedules of reinforcement (e.g., differential reinforcement of low rate behavior versus differential reinforcement of high rate behavior) may produce differences in response rates, but it is not clear that these differences should be attributed to differences in response strength (see Nevin, 1974).

Support for the characterization of response strength provided by behavioral momentum theory and for the proposed separable roles of the response–reinforcer and stimulus–reinforcer relations comes from experiments arranging additional response-independent reinforcers in one component of a multiple schedule. The inclusion of added response-independent reinforcers degrades the response–reinforcer relation but improves the stimulus–reinforcer relation. For example, Nevin, Tota, Torquato, and Shull (1990) showed that adding response-independent reinforcers to one component of a two-component multiple schedule reduced pigeons' response rates in that component, but nevertheless increased response strength as measured with tests of resistance to change. Similar findings have been obtained in a number of related experiments using different types of subjects and reinforcers (Ahearn, Clark, Gardenier, Chung, & Dube, 2003; Cohen, 1996; Grimes & Shull, 2001; Harper, 1999; Igaki & Sakagami, 2004; Mace, Lalli, Shea, Lalli, West, Roberts, et al., 1990; Nevin et al., 1990; Podlesnik & Shahan, 2008; Shahan & Burke, 2004).

The vast majority of research on resistance to change has been conducted on behavior maintained by primary reinforcement. Relatively few experiments have examined resistance to change of responding maintained by conditioned reinforcers. Shahan, Magee, and Dobberstein (2003) examined resistance to change of responding maintained by conditioned reinforcers using a multiple schedule of observing-response procedures with pigeons. In each of two components of a multiple schedule, an observing-response procedure was arranged using different stimuli. Both components arranged a mixed schedule in which nondifferentially signaled periods of exposure to a random-interval (RI) schedule of food reinforcement for responses to a key (i.e., the food key) alternated unpredictably with extinction. Responses to a second key (i.e., the observing key) produced brief periods of exposure to stimuli differentially signaling the RI schedule (i.e., S+) or extinction on the food key. Observing responses are generally believed to be maintained by the conditioned reinforcing effects of S+ presentations (see Dinsmoor, 1983; Fantino, 1977, for reviews). The two components in Shahan et al. differed in the rate of primary reinforcement arranged by the RI schedule. Thus, observing responses in one component produced an S+ associated with a higher rate of primary reinforcement (e.g., RI 15 s) and in the other component produced an S+ associated with a lower rate of primary reinforcement (e.g., RI 60 s). Shahan et al. found that observing rates were higher and more resistant to change in the component with an S+ associated with a higher rate of primary reinforcement. This result was comparable to the outcome of a similar examination of the effects of terminal-link reinforcement rate on initial-link response rates and resistance to change in a multiple schedule of two-link chain schedules (Nevin, Mandell, & Yarensky, 1981).

It is important to note, however, that Shahan et al. (2003) arranged different rates of primary reinforcement in the two observing-response components. To further examine the applicability of behavioral momentum theory to responding maintained by conditioned reinforcement, Shahan and Podlesnik (2005) examined the effects of differences in rates of conditioned reinforcement on observing rates and resistance to change. A multiple schedule of observing response procedures similar to that in Shahan et al. was used. In contrast to Shahan et al., the arranged rate of primary reinforcement in the two observing-response components was the same and the rate of conditioned reinforcement differed. Observing responses in both components produced an S+ differentially associated with the availability of food, but S+ was delivered at different rates in the two components by arranging different RI schedules on the observing keys. In Experiment 1, observing responses produced an S+ on an RI 15-s schedule in one component and on an RI 60-s schedule in the other component. In Experiment 2, an S+ was produced on an RI 10-s schedule in the rich component and on an RI 60-s schedule in the lean component. In both experiments, observing rates were higher in the component associated with the higher rate of conditioned reinforcement. However, resistance to change of observing was not impacted by differential rates of conditioned reinforcement in the two components. In fact, in Experiment 2 observing was somewhat more resistant to change in the component arranging the lower rate of conditioned reinforcement. This result likely occurred because the more extreme parameters arranged in Experiment 2 resulted in somewhat higher obtained rates of primary reinforcement in the component with the lower conditioned reinforcement rate. Thus, resistance to change of responding maintained by conditioned reinforcement appeared to differ only when there was a difference in rates of primary reinforcement in the components.

The findings of Shahan and Podlesnik (2005) suggest that parameters of conditioned reinforcement may impact baseline response rates but not resistance to change. Shahan et al. (2003) previously attributed the differences in resistance to change they obtained with S+ stimuli associated with different rates of primary reinforcement to differences in the conditioned reinforcing value of the S+ deliveries. However, the findings of Shahan and Podlesnik suggest that parameters of conditioned reinforcement may have no impact on resistance to change (i.e., response strength) independent of the effects of differences in primary reinforcement. The present experiments directly evaluated the effects of rates of primary reinforcement and value of conditioned reinforcers on observing-response rates and resistance to change.

Experiment 1

This experiment placed conditioned reinforcement value and rate of primary reinforcement in opposition to one another and examined their effects on observing rates and resistance to change. In a multiple schedule of observing-response procedures, observing in both components was maintained by the same rate of S+ presentation. In addition, food-key responses were maintained by the same rate of response-dependent primary reinforcement. In one component, additional response-independent food deliveries uncorrelated with the conditions of response-dependent reinforcement on the food key were also delivered. Response-independent food deliveries could occur during S+ or anytime during the mixed schedule on the food key (i.e., during a VI schedule or during extinction). The added food deliveries increased the overall rate of primary reinforcement, but were expected to decrease the value of the S+ deliveries because they degraded the predictive relation between S+ and food deliveries.

Method

Subjects

The subjects were 3 White Carneau pigeons and 1 homing pigeon maintained at approximately 80% of their free-feeding weights (+/− 15g) by postsession supplemental feeding as necessary. Pigeons 77, 210, 206, and 38 (homing pigeon) weighed 466 g, 430 g, 419 g, and 334 g, respectively. The pigeons varied in age and had extensive experience in other experiments. When not in the experimental sessions, the pigeons were housed in individual cages in a temperature-controlled colony with a 12:12 hr light/dark cycle (lights on at 7:00 a.m.) and had free access to water.

Apparatus

The experiment was conducted in four Lehigh Valley Electronics pigeon chambers measuring 350 mm long, 350 mm high, and 300 mm wide. Three response keys were centered on the front panel 83 mm apart (center to center) and were 240 mm above the floor. The keys measured 25 mm in diameter, and required about 0.1 N to operate. A projector could transilluminate each key with red, green, blue, yellow, a white plus sign with a black background, a white triangle with a black background, a white vertical line with a black background, or a white horizontal line with a black background. All recorded responses turned off the houselight for 0.01 s to provide response feedback. Reinforcers consisted of 2-s presentations of pigeon checkers from a hopper. The hopper was accessible, when raised, through a 50 mm wide by 55 mm tall aperture located on the midline of the work panel with its center 100 mm from the floor. A 28-V DC clear bulb illuminated the aperture, and all other lights were extinguished when the hopper was operated. General illumination was provided by a shielded 28-V DC clear bulb mounted 45 mm above the center key. A ventilation fan and white noise masked extraneous sounds. Control of experimental events and data recording were conducted with Med Associates® programming and interfacing.

Procedure

Overview

All pigeons had participated in previous experiments using similar procedures and started immediately on the baseline multiple-schedule of observing-response procedures. Figure 1 shows a schematic of the procedure. Following a 30-s blackout, one of two multiple-schedule components arranging an observing-response procedure was randomly selected to begin each session. Components strictly alternated thereafter and were 5 min in duration. Components were separated by a 30-s intercomponent interval (ICI) during which the houselight and keylights were off. Sessions ended after a total of four presentations of each component.

Fig 1.

Diagram of procedure for Experiment 1. See text for details.

In both observing-response procedures, periods of exposure to a VI 60-s schedule of reinforcement and extinction alternated on the center key after variable times averaging 60 s (ranging from 10 to 110 s in 10-s increments). Nondifferential mixed-schedule stimuli were presented on the center (i.e., food) and left (i.e., observing) keys regardless of whether the VI 60-s schedule or extinction was active on the food key. The right key was always dark and inoperative. Responses on the observing key occasionally changed the mixed-schedule stimuli on both keys to S+ for 15 s (exclusive of hopper time). Responses to the observing key during S+ presentations had no programmed consequences. Appendix A provides the mixed-schedule and S+ stimuli for each pigeon. Observing responses produced S+ presentations when (a) the VI component was in effect on the food key, and (b) the selected interval from a VI 15-s schedule on the observing key had elapsed. Timing of the VI 15-s schedule on the observing key was halted while extinction was in effect on the center key and no S- stimulus was ever presented (cf. Dinsmoor, Browne, & Lawrence, 1972; Dinsmoor, Mulvaney, & Jwaideh, 1981; Shahan, Podlesnik, & Jimenez-Gomez, 2006).

Scheduled rates of S+ presentation and food reinforcement were the same in the two observing-response components; however, response-independent food presentations uncorrelated with the VI and extinction periods were added to one component on a VT 45-s schedule. Because these uncorrelated food presentations would be expected to decrease the value of S+ presentations, we refer to the component with the added uncorrelated food deliveries as the Low-Value component and the component without the added food as the High-Value component. Response-independent food presentations could occur at any time during the mixed or S+ stimuli of the Low-Value component. Figure 2 shows how the added VT food deliveries impacted food delivery rates and expected value of S+ deliveries in the two components. The top panel shows average obtained rates of food delivery across the baseline conditions. As a result of the uncorrelated VT food deliveries, the overall rate of food was higher in the Low-Value component. The higher overall food-delivery rates resulted from higher rates during both S+ and the mixed-schedule stimuli. We will calculate the expected value of S+ in two ways. The first is based on the improvement in food-delivery rate signaled by the transition from the mixed-schedule stimuli to S+ (i.e., the ratio of obtained food delivery rates during S+ to those obtained during the mixed schedule). The second is the average reduction in delay to food signaled by the transition from the mixed schedule to S+ (see Fantino, 1977, for review). For example, in the High-Value component, transitioning from the mixed stimulus (0.2 food deliveries obtained per min—a 300 s average interfood interval) to S+ (55 s obtained interfood interval) signaled a 245-s delay reduction. The bottom of Figure 2 shows that expected value was higher in the High-Value component than the Low-Value component regardless of the method by which it was calculated. Thus, as anticipated, the Low-Value component arranged higher rates of food delivery but was associated with an S+ with a lower expected value.

Fig 2.

Food deliveries per minute calculated for each component as a whole (i.e., Overall) and calculated separately for reinforcers delivered in the presence of S+ and the mixed-schedule stimuli. The bottom of the figure shows expected value of the S+ in each component calculated as the ratio of reinforcement rates in S+ to those in the mixed-schedule or as the reduction in delay to food signaled by S+.

Procedural Details

In both components, transitions from the VI schedule to extinction on the food key during S+ presentations turned off S+ and turned on the mixed-schedule stimuli. S+ deliveries arranged during a VI period on the food key but not earned before a transition to extinction on the food key were held until the next VI component was available, but were cancelled with transitions between the two observing-response components. Similarly, undelivered reinforcers arranged by the VI schedule on the food key were held until the VI schedule was presented again within a particular observing-response component, but were canceled with transitions between the components. Schedule transitions could not occur during hopper presentations, which were excluded from all timing throughout the experiment. A changeover delay (COD) prevented responses on the food key from producing food deliveries within 3 s of a response to the observing key. All VI and VT food schedules used throughout the experiment were arranged according to Fleschler and Hoffman (1962) progressions with 10 intervals. Sessions occurred at approximately the same time each day and typically were conducted 7 days per week. These conditions remained in effect until observing rates and food-key response rates during S+ were visually judged to be stable with no increasing or decreasing trends across at least six sessions.

Resistance to Change Tests

Following stability, resistance to change was assessed by examining the effects of presession feeding (Prefeeding) and Extinction. For Prefeeding, the pigeons were fed 30 g of pigeon checkers in the home cage 1 hr prior to five consecutive sessions (cf. Shahan & Podlesnik, 2005). Following Prefeeding, running weights were reestablished and baseline reestablished for at least six sessions. Next, the pigeons were exposed to five consecutive sessions of Extinction in which observing did not produce S+ and food was not presented for food-key responses. Therefore, only mixed-schedule stimuli appeared throughout extinction conditions.

Observing-response rates were calculated by dividing the number of observing responses during the mixed schedule by time spent in the presence of the mixed-schedule stimulus. Rates of responding to the food key in the presence of S+ and the mixed-schedule stimulus were also calculated separately. Proportion of baseline response rates during disruption was used as the measure of resistance to change. Proportion of baseline was calculated by dividing response rates in each session of disruption by the predisruption baseline response rates in the immediately preceding baseline.

Appendix B shows the number of sessions in baseline prior to each condition of disruption, mean response rates from the last six sessions of each baseline condition, and response rates from individual sessions of disruption.

Results

The top panel of Figure 3 shows baseline observing rates in the High- and Low-Value components across the two baseline conditions for individual pigeons. Data represent means of the last six sessions of exposure to each baseline. Observing rates were either similar in the two components (Pigeons 77 and 210) or higher in the High-Value component than in the Low-Value component (Pigeons 206 and 38). The bottom panel shows food-key response rates in the High- and Low-Value components in the presence of S+ and in the presence of the mixed-schedule stimulus. Within the components, there was some tendency for food-key response rates to be higher during S+ than during the mixed-schedule stimulus, although this relation was somewhat less clear for Pigeon 77 in the High Value component and was completely absent for Pigeon 38. Comparing across the two components there was no reliable difference in food-key response rates during the mixed schedule or during S+.

Fig 3.

Response rates on the observing key and food key in the High-Value and Low-Value components across successive exposures to the baseline condition. Response rates on the food key are presented separately for responding in the presence of S+ and the mixed-schedule stimuli. The y axes are logarithmic. Data represent the means of the last six sessions of exposure to the conditions. Error bars represent ±1 SD.

The top panel of Figure 4 shows observing response rates as a proportion of the predisruption baseline rate for disruption by Prefeeding and Extinction. Observing was more resistant to change in the Low-Value component than in the High-Value component, with the only exception being Pigeon 38 in the Prefeeding condition. Similarly, the middle panel shows that food-key responding in the presence of the mixed-schedule stimulus was also more resistant to change in the Low-Value component than in the High-Value component, with the only exception being Pigeon 210 in the Prefeeding condition. Finally, the bottom panel shows that resistance to change of food-key responding in the presence of S+ was not systematically different for the two components. Resistance to change of S+ food-key responding during extinction could not be assessed because S+ was not presented during Extinction.

Fig 4.

Response rates in the High- and Low-Value components during disruption by Prefeeding (i.e., PF) and Extinction (i.e., E) as a proportion of the immediately preceding baseline conditions. Data for observing responses, food-key responses in the presence of the mixed schedule, and food-key responses in the presence of S+ are presented in the top, middle, and bottom panels, respectively.

Discussion

Food deliveries uncorrelated with the conditions of primary reinforcement on the food key resulted in lower observing rates for 2 of 4 subjects. The uncorrelated food deliveries had little systematic effect on food-key responding. Nonetheless, both observing and food-key responding during the mixed schedule were more resistant to change in the component with the added food deliveries. The added food deliveries had no consistent effect on resistance to change of responding on the food key in the presence of S+. This difference in the effects of the added food on observing and mixed-schedule food-key responding on the one hand, and S+ food-key responding on the other, is not surprising given the differences in obtained relative food-delivery rates for the two components during the mixed schedule and during S+. The uncorrelated food deliveries in the Low-Value component increased food rates during both the mixed schedule and during S+, but the increase relative to the food rates in the High-Value component was considerably larger during the mixed schedule than during S+. The across-component ratio of food delivery rates during the mixed schedule in the Low-Value versus High-Value components was 7.5 (i.e., 1.5/0.2, see Figure 2). It is important to note that the mixed-schedule stimulus was the context in which both observing and mixed-schedule food-key responding occurred. In contrast, the ratio of food delivery rates during S+ in the two components was only 2.0 (i.e., 2.3/1.1). The smaller across-component ratio of reinforcement rates in S+ resulted from the fact that the added food deliveries contributed relatively less to the already high rates of food delivery arranged by S+. Thus, the reliable effects of the added food deliveries on observing and food-key responding during the mixed schedule likely reflected the relatively greater rates of food delivery obtained in the presence of the mixed-schedule stimulus in the Low-Value component.

The uncorrelated food deliveries were added to the Low-Value component in an attempt to decrease the value of S+ deliveries in that component while simultaneously increasing rates of primary reinforcement. Given the obtained rates of food delivery in the presence of S+ relative to those in the presence of the mixed-schedule stimulus, one would expect that the value of S+ deliveries was indeed decreased in the Low-Value component. However, the fact that observing rates were decreased for only 2 of 4 pigeons raises some doubt about the success of the attempt to decrease the value of S+. Regardless, it is notable that resistance to change of observing was greater in the Low-Value component for pigeons showing lower observing rates in that component and for pigeons showing no difference in observing rates in the two components. Thus, the rate of primary reinforcement appears to have had a relatively large impact on resistance to change regardless of the inconsistent effects of S+ value on response rates.

In Experiment 2 we sought a more straightforward method for comparing the effects of conditioned reinforcement value and primary reinforcement rate on observing rates and resistance to change.

Experiment 2

In the observing-response procedure, unsignaled periods of availability of some schedule of primary reinforcement (e.g., VI) typically alternate with periods of extinction. In a number of previous experiments using observing and related procedures, the periods of reinforcement and extinction have been presented probabilistically and the probability of a reinforcement period has been varied (e.g., Eckerman, 1973; Hendry, 1965; Kendall, 1973; McMicheal, Lanzetta, & Driscoll, 1967; McMillian 1974; Wilton & Clements, 1971). When the probability of a reinforcement period is decreased the probability that observing will produce an S+ is also decreased. A general finding from these experiments is that observing is an asymmetrical inverted U-shaped function of probability of a reinforcement period (i.e., probability of S+). When the probability of a reinforcement period is increased above p  =  .5, observing decreases. When the probability of reinforcement period decreases below p  =  .5, observing first increases and then decreases at lower probabilities. Thus, although both rates of primary reinforcement and rates of S+ delivery are decreased when the probability of a reinforcement period is decreased below p  =  .5, observing nonetheless increases. One widely accepted interpretation of this outcome is that decreases in the probability of a reinforcement period increase the conditioned reinforcing value of S+ presentations (see Dinsmoor, 1983; Fantino, 1977, for reviews). The increase in S+ value is thought to result from the relatively larger improvement in primary reinforcement rate or reduction in delay to primary reinforcement signaled by S+ relative to that signaled by the mixed-schedule stimulus as the probability of a reinforcement period decreases. In other words, when reinforcement periods are more probable than extinction periods in the mixed schedule, the reinforcement rate signaled by the mixed schedule is closer to that arranged during S+ than when reinforcement periods are less probable.

Given that decreases in the probability of a reinforcement period during the mixed schedule decrease primary reinforcement rates but apparently increase the value of S+ presentations, such a manipulation provides an interesting way to examine the relative impact of primary reinforcement rate and conditioned reinforcement value on resistance to change. In this experiment, a multiple schedule of observing-response procedures was arranged in which the two components differed in terms of the probability of a VI versus extinction period during the mixed schedule. In one component, the probability of a VI period was .6 and in the other the probability of a VI period was .1. Thus, the conditioned reinforcing value of the S+ in the component with p  =  .6 of a VI period would be expected to be lower than for the S+ in the component with p  =  .1 of a VI period. Based on the previous research described above, we anticipated that observing rates would be higher in the component with the lower probability of a VI period (i.e., greater conditioned reinforcement value). Nonetheless, the arranged rate of primary reinforcement is six times lower in the component with the lower probability of a VI period, and based on Experiment 1 observing might be expected to be less resistant to change in that component.

Method

Subjects

The subjects were 4 homing pigeons maintained under the same conditions as those in Experiment 1. Pigeons 225, 270, 622, and 199 weighed 303 g, 336 g, 386 g, and 408 g, respectively.

Apparatus

The experiment was conducted in four Lehigh Valley Electronics pigeon chambers similar to those described in Experiment 1. The keys could be transilluminated red, blue, turquoise, and yellow.

Procedure

All pigeons had participated in previous experiments using similar procedures and started immediately on the baseline multiple schedule of observing-response procedures. Except where noted, the details of the procedure were as in Experiment 1. In both observing-response components, nondifferential mixed-schedule stimuli were presented on the center (i.e., food) and left (i.e., observing) keys regardless of whether a VI 30-s schedule or extinction was active on the food key. Appendix A provides the mixed-schedule and S+ stimuli for each pigeon. The VI schedule and extinction periods were arranged probabilistically rather than strictly alternating as in Experiment 1. The mean duration of the VI and extinction periods was 30 s and ranged from 10 s to 50 s in 10-s increments. The probability of a VI period was greater for one observing-response component than for the other. Because the value of S+ deliveries would be expected to be higher in the observing-response component with less frequent VI periods, we will refer to the component with the less frequent VI periods as the High-Value component and the component with the more frequent VI periods as the Low-Value component. In the High-Value component, a VI period occurred with p  =  .1 and extinction periods occurred with p  =  .9. In the Low-Value component, a VI period occurred with p  =  .6 and extinction periods occurred with p  =  .4. As in Experiment 1, observing responses produced S+ presentations when (a) the VI component was in effect on the center key, and (b) the selected interval from a VI 15-s schedule on the observing key had elapsed. All other details of the schedule operations also were as in Experiment 1. These conditions were in effect until performance was deemed stable as assessed in Experiment 1.

Figure 5 shows mean obtained rates of food delivery in the baseline conditions for the two components. Overall obtained food delivery rates were calculated separately for time in the presence of S+ and the mixed-schedule stimulus. Overall and mixed-schedule food delivery rates were higher in the Low-Value component than in the High-Value. Rates of food delivery in the presence of S+ were similar for the two components. The bottom of Figure 5 shows the impact of these food delivery rates on expected value of the S+ in both components. Value, calculated either as the ratio of food rates in S+ relative to the mixed schedule or as the obtained reduction in delay to primary reinforcement signaled by S+, was higher in the High-Value component than in the Low-Value component. Given the similar food rates during S+ in the two components, the difference in value resulted largely from the lower rate of mixed-schedule food delivery in the High-Value component. Also, as expected, the obtained rate of S+ delivery was higher in the Low-Value component (M  =  1.53, SD  =  0.21) than in the High-Value component (M  =  0.22, SD  =  0.02).

Fig 5.

Food deliveries per minute for the different stimulus conditions and expected value of S+ deliveries in the High- and Low-Value components in Experiment 2. Details are as in Figure 2.

Following stability, resistance to change was assessed with a number of disrupters. Following each condition of disruption, baseline was reestablished for at least 6 consecutive sessions and until stable performance again was observed. First, pigeons were fed 40 g of pigeon checkers 1 hr prior to 5 consecutive sessions (Prefeeding). Second, response-independent food was presented during the ICI on a fixed-time (FT) 0.75-s schedule while food and S+ presentations were simultaneously discontinued for 5 consecutive sessions (Extinction + ICI FT Food). Third, response-independent food was presented during the ICI on a FT 0.75-s schedule for 5 consecutive sessions (ICI FT Food). Fourth, food and S+ presentations were discontinued for 10 consecutive sessions (Extinction). Fifth, food presentations were discontinued, but observing continued to produce S+ presentations for 20 consecutive sessions (Extinction with S+). The latter condition was continued for 20 sessions because 10 sessions of exposure was not enough to produce reliable decreases in responding in the Low-Value component for 2 pigeons. Pigeon 199 was exposed to only 15 sessions of Extinction with S+ because response rates had decreased to near zero. The order of exposure to ICI FT Food and Extinction was reversed for Pigeon 270. Appendix C shows the number of sessions in baseline and disruption conditions, mean response rates from the last 6 sessions from each baseline condition, and response rates from individual sessions of disruption.

Results

The top panel of Figure 6 shows baseline observing rates across successive baseline conditions. Observing rates were higher in the High-Value component than in the Low-Value component for all baselines and for all pigeons except 199. The bottom panel of Figure 6 shows food-key response rates in the presence of S+ and the mixed-schedule stimulus. In both the High- and Low-Value components, response rates were typically higher in S+ than in the mixed-schedule stimulus. Response rates during S+ were similar in the High- and Low-Value components, but response rates in the mixed-schedule stimulus were lower in the High-Value component than in the Low-Value component. As a result, the difference between S+ and mixed-schedule response rates was greater in the High-Value component than in the Low-Value component.

Fig 6.

Response rates on the observing key and food key in the High-Value and Low-Value components across successive exposures to the baseline conditions in Experiment 2. Data are presented as in Figure 3.

Figure 7 shows observing and food-key responding during disruption by Prefeeding (PF), Extinction+ICI FT Food (Eft), and ICI FT Food (FT) as a proportion of baseline rates. The top panel shows that observing was more resistant to change in the Low-Value component than in the High-Value component for all pigeons and disruptors. The middle panel shows that food-key responding in the presence of the mixed-schedule stimulus also tended to be more resistant to change in the Low-Value component than in the High-Value component. Three exceptions to this finding were for Pigeon 270 with ICI FT Food and Pigeon 622 for both Prefeeding and ICI FT Food. The bottom panel shows resistance to change of food-key responding in the presence of S+ was not consistently different across the two components when considered across the two disruptors.

Fig 7.

Response rates in the High- and Low-Value components during disruption by Prefeeding (i.e., PF), ICI FT food (i.e., FT), and Extinction+FT (i.e., Eft) as a proportion of the immediately preceding baseline conditions in Experiment 2. Data are presented as in Figure 4.

Figure 8 shows observing and food-key responding as a proportion of baseline during five-session blocks of disruption by Extinction and Extinction with S+. Resistance to change of observing during the two 5-session blocks of exposure to Extinction (i.e., triangles in left panels) was greater in the Low-Value than in the High-Value component in seven of eight cases. This difference was smaller and less consistent for Pigeons 225 and 199. Similarly, with the exception of Pigeon 225, food-key responding during the mixed schedule (i.e., triangles in the middle panels) was more resistant to disruption by Extinction in the Low-Value component than in the High-Value component. Data for responding in S+ during disruption by Extinction are not presented in the right panels because S+ was never presented during that disruptor.

Fig 8.

Response rates during five-session blocks of disruption by Extinction (i.e., Ext) and Extinction with S+ (i.e., Ext S+) as a proportion of baseline rates in the High- and Low-Value components. Data for observing responses, food-key responses in the presence of the mixed schedule, and food-key responses in the presence of S+ are presented in the left, middle, and right panels, respectively.

Resistance to change of observing during Extinction with S+ available (circles in the left panels of Figure 8) was greater in the Low-Value component than in the High-Value component for at least two 5-session blocks. However, resistance to change of food-key responding during the mixed-schedule and during S+ (circles in the middle and right panels, respectively) did not differ consistently between the Low- and High-Value components. Finally, when comparing only the first two 5-session blocks, observing was always more resistant to change during Extinction with S+ than during Extinction (compare empty circles to empty triangles and filled circles to filled triangles). Thus, observing was more resistant to change when it continued to produce S+ than when it did not.

Discussion

Observing response rates were higher in the High-Value component in which the probability of a reinforcement period was lower. This outcome occurred despite the fact that the rates of primary reinforcement and S+ presentation were lower in the High-Value component. A number of previous experiments manipulating the probability of a reinforcement period have obtained a similar result (e.g., Eckerman, 1973; Hendry, 1965; Kendall, 1973; McMicheal et al., 1967; McMillian 1974; Wilton & Clements, 1971). Higher observing rates with less frequent periods of primary reinforcement have typically been attributed to greater conditioned reinforcing value of S+ presentations (e.g., Fantino, 1977). Despite the higher observing rates and likely greater conditioned reinforcing value of S+ in the High-Value component, observing was more resistant to change in the Low-Value component. Although the rate of S+ delivery was higher in the Low-Value component in the present experiment, Shahan and Podlesnik (2005) showed that rate of S+ delivery had no effect on resistance to change when rates of primary reinforcement were controlled. Thus, the greater resistance to change in the Low-Value component likely resulted from the higher rate of primary reinforcement in that component.

The results of this experiment and Experiment 1 suggest that when primary reinforcement rate and conditioned reinforcing value of S+ are placed in opposition, resistance to change of observing is governed by primary reinforcement rate. One interpretation of this finding is that conditioned reinforcement value does not affect resistance to change. Alternatively, the impact of conditioned reinforcement value on resistance to change may have been overwhelmed by the higher primary reinforcement rate in the Low-Value component. To evaluate these two hypotheses, Experiment 3 examined the effects of conditioned reinforcement value on observing rates and resistance to change when equal rates of primary reinforcement were arranged in the two components.

Experiment 3

This experiment used a procedure like that in Experiment 2 in which two components of a multiple schedule of observing-response procedures arranged different probabilities of a reinforcement period. However, the programmed rate of primary reinforcement in the two components was the same. To accomplish this, the rate of primary reinforcement was increased during the reinforcement periods in the High-Value component and decreased in the Low-Value component. The VI schedules on the food key in the two components were set at values chosen to counteract the six-fold difference in their frequency of occurrence while still arranging differences in the expected value of S+ presentation.

Method

Subjects

The 4 homing pigeons from Experiment 2 were used.

Apparatus

The experimental chambers were those used in Experiment 2 and three of the chambers used in Experiment 1, as detailed below.

Procedure

All pigeons began on the final procedure. Except where noted, the procedure was identical to that used in Experiment 2. For 3 pigeons (225, 270, and 622), the experiment was conducted twice because the stimulus assignments used in the first exposure to the conditions were the same as arranged in Experiment 2. The condition replications were conducted to verify that the results were not a product of previous experience with those key colors. For the replication, these pigeons were moved to the chambers used in Experiment 1 and a different stimulus set was used (see Appendix A). We refer to the first exposure to the conditions of this experiment as Part 1 and the replication as Part 2. The 4th pigeon (Pigeon 199) was not exposed to Part 2 because it had fallen considerably behind the other pigeons during Experiment 2. Given the outcome with the other pigeons in Parts 1 and 2, it was clear that there was no pattern of behavior that Pigeon 199 could produce in Part 2 that would meaningfully alter the final conclusions.

In the Low-Value component, VI 51.43-s periods occurred with p  =  .6 and extinction periods occurred with p  =  .4. In the High-Value component, VI 8.57-s periods occurred with p  =  .1 and extinction periods occurred with p  =  .9. Thus, programmed overall food delivery rates were equated for the two components at 0.7 foods/min [i.e., food/min  =  60/(VI value/p of VI period)]. Observing responses changed the mixed-schedule stimuli to S+ on a VI 15-s schedule in both components. As in Experiments 1 and 2, observing responses produced S+ presentations when (a) the VI component was in effect on the center key, and (b) the selected interval from a VI 15-s schedule on the observing key had elapsed. Transitions from the VI schedule to extinction during a 15-s S+ presentation turned off S+ and turned on the mixed-schedule stimuli. These conditions remained in effect until observing and S+ response rates on the food key appeared stable in both components across at least six sessions.

Figure 9 shows obtained food delivery rates in Parts 1 and 2. Obtained overall and mixed-schedule food delivery rates were similar in the High-Value and Low-Value components, but were slightly higher in the Low-Value component. As expected based on the different VI schedules arranged, food delivery rates were considerably higher during S+ in the High-Value component. The bottom portion of each panel of Figure 9 shows the impact of these food delivery rates on expected value of the S+ in both components. Despite the similar overall and mixed-schedule food rates, value, calculated either as the ratio of food rates in S+ relative to the mixed schedule or as the obtained reduction in delay to primary reinforcement signaled by S+ , was considerably greater in the High-Value component. Thus, the procedure succeeded in arranging similar overall food delivery rates with different valued S+ stimuli in the two components.

Fig 9.

Food deliveries per minute for the different stimulus conditions and expected value of S+ deliveries in the High- and Low-Value components for Parts 1 and 2 of Experiment 3.

Once responding was stable in both components, the effects of Prefeeding 40g of pigeon checkers, ICI FT Food, and Extinction with S+ were examined as described for Experiment 2. Following each condition of disruption, the baseline was reestablished for at least six consecutive sessions and until stable performance again was observed. Appendices 4 and 5 show the number of sessions in each baseline, mean response rates from the last six sessions from each baseline condition, and response rates from individual sessions of disruption for Parts 1 and 2.

Results

The top panels of Figure 10 show observing in the High- and Low-Value components across successive baseline conditions for Part 1 (left panel) and Part 2 (right panel). Observing rates were typically higher in the High-Value component than in the Low-Value component. The difference was relatively small for Pigeons 622 and 199 in Part 1 and reversed for Pigeon 622 in Part 2. The bottom panels of Figure 10 show food-key response rates in the mixed-schedule stimulus and in S+. In both components, response rates in S+ tended to be higher than in the mixed-schedule stimulus. Response rates during S+ were not systematically different across the two components. Mixed-schedule response rates tended to be higher during the Low-Value component than during the High-Value component.

Fig 10.

Response rates on the observing key and food key in the High-Value and Low-Value components across successive exposures to the baseline conditions for Parts 1 and 2 of Experiment 3. The left and right panels are for data from Part 1 and 2, respectively.

Figure 11 shows observing and food-key responding during disruption by Prefeeding and ICI FT Food as a proportion of baseline rates. Data for Parts 1 and 2 are presented in the left and right panels, respectively. There was some tendency for observing to be more resistant to change in the Low-Value component than in the High-Value component, although there were a number of exceptions. Observing was more resistant to change in the High-Value component for Pigeon 199 for both disruptors in Part 1. In addition, for Pigeon 622, observing was not reliably disrupted by ICI FT Food for either component in Part 1 and was more resistant to change in the High-Value component with ICI FT Food in Part 2. Resistance to change of food-key responding during the mixed schedule was not systematically different for the High- and Low-Value components across disruptors in Parts 1 and 2. There was a tendency for food-key responding during S+ to be more resistant to change in the High-Value component than in the Low-Value component, but there were two cases where neither component showed any disruption (i.e., Prefeeding for Pigeons 225 and 199 in Part 1) and three cases where the responding in the Low-Value component was more resistant to change (ICI FT Food for Pigeon 270 in Part 1, Prefeeding for Pigeon 622 in Part 1 and for Pigeon 225 in Part 2).

Fig 11.

Response rates in the High- and Low-Value components during disruption by Prefeeding (i.e., PF) and ICI FT food (i.e., FT) as a proportion of the immediately preceding baseline conditions in Parts 1 and 2 of Experiment 3. The left and right columns are for data from Part 1 and 2, respectively.

Figure 12 shows observing and food-key responding during five-session blocks of disruption by Extinction with S+ in Parts 1 and 2. The left and center panels show that resistance to change of neither observing nor food-key responding during the mixed schedule were consistently different for the High-Value and Low-Value components. There was some tendency for resistance to change of observing to be greater in the Low-Value component and resistance to change of food-key responding to be greater in the High-Value component, but there were a number of exceptions in both cases. Resistance to change of food-key responding during S+ was typically greater in the High-Value component than in the Low-Value component.

Fig 12.

Observing and food-key responding during five-session blocks of disruption by Extinction with S+ presented as a proportion of baseline rates in the High- and Low-Value components in Parts 1 and 2 of Experiment 3. Data for observing responses, food-key responses in the presence of the mixed schedule, and food-key responses in the presence of S+ are presented in the left, middle, and right panels, respectively.

Discussion

As in Experiment 2, observing response rates were typically higher in the High-Value component than in the Low-Value component. Nonetheless, there was a tendency for resistance to change of observing to be greater in the Low-Value component. This tendency was not entirely consistent across pigeons and disruptors and this may reflect the somewhat higher obtained overall rate of primary reinforcement in the Low-Value component (see Figure 9). Thus, even when similar rates of primary reinforcement are arranged, the value of a conditioned reinforcer appears to have no detectable effect on resistance to change.

General Discussion

The present experiments examined the impact of conditioned reinforcement value on observing response rates and resistance to change. In Experiment 1, conditioned reinforcement value was decreased in one component by adding food deliveries uncorrelated with the reinforcement conditions signaled by S+. In Experiments 2 and 3, the value of S+ was increased in one component by decreasing the probability of VI food-reinforcement periods signaled by S+. Figure 13 shows a summary of the effects of differences in value of S+ on observing rates in each of the three experiments. The left panel shows mean observing rates across predisruption baseline conditions as a function of S+ value calculated as the ratio of food delivery rates in S+ and the mixed-schedule stimulus. The right panel similarly shows mean baseline observing rates as a function of S+ value calculated as the reduction in delay to food signaled by the onset of S+. In each experiment, average observing rates were a positive function of the value of S+ calculated in either way, and observing rates were always significantly higher for the higher-valued S+ than the lower-valued S+ [ts(7-20) > 2.4, all p < .05].

Fig 13.

Mean observing response rates for the baseline conditions of each experiment plotted as a function of S+ value calculated as either the ratio of food reinforcers obtained in the presence of S+ and the mixed-schedule stimulus (i.e., left panel) or the reduction in delay to food reinforcement signaled by S+ (i.e., right panel). Both axes are logarithmic.

Despite the impact of S+ value on observing rates, resistance to change was not systematically related to S+ value. This outcome occurred when rates of primary reinforcement and value of S+ were placed in opposition to one another in Experiments 1 and 2, and when primary reinforcement rates were similar in both parts of Experiment 3. The top panels of Figure 14 summarize the relation between relative resistance to change and relative S+ value in each of the present experiments. The fitted function is based on a statement of behavioral momentum theory (e.g., Grace & Nevin, 1997) relating relative resistance to change to relative reinforcement rate such that:

1

where Bx and Bo refer to response rates during disruption and baseline, respectively, and R refers to reinforcement rates. The subscripts denote the two components of a multiple schedule. The parameters a and logb refer to sensitivity of relative resistance to change to variations in the ratio of reinforcement rates provided in the two components and bias in relative resistance to change unrelated to the reinforcement ratio, respectively. For the fits of Equation 1 to the data in the top panels of Figure 14, mean S+ value in the two components has been substituted for R₁ and R₂. The left panel uses value calculated as the ratio of reinforcement rates in S+ and the mixed-schedule stimulus. The right panel uses value calculated as the reduction in delay to primary reinforcement signaled by the onset of S+. In every case, relative resistance to change was greater in the Low-Value component than in the High-Value component (y-axis values < 0). The functions are negative for both ways of calculating value, but the slopes of neither function are statistically different from zero. Thus, relative resistance to change does not appear to be an orderly function of variations in the relative value of S+. If anything, resistance to change is greater for responding maintained by an S+ with a lower value.

Fig 14.

The top panels show relative resistance to change of observing in the High-Value and Low-Value components as a function of the log ratio of S+ value in the two components. Data are means across subjects and disruptors for each of the experiments. The bottom panels show relative resistance to change of observing as a function of the log ratio of food delivery rates. Relative resistance to change in the bottom panels is calculated as resistance to change in the component with the higher obtained rate of food delivery minus resistance to change in the component with the lower obtained rate of food delivery. The ratio of food delivery rates in the bottom panels was constructed with the component associated with the higher rate of food delivery always in the numerator. Regressions of Equation 1 are shown with parameter values and variance accounted for (VAC).

On the other hand, the bottom panels of Figure 14 show that resistance to change was an orderly function of rates of primary reinforcement in the multiple-schedule components. The left panel shows relative resistance to change as a function of obtained rates of food delivery in the presence of the mixed-schedule stimuli. The right panel shows relative resistance to change as a function of overall obtained rates of food delivery in the two components. For both panels, relative resistance was calculated by subtracting resistance to change in the component with the lower obtained rate of food, from resistance to change in the component with the higher obtained rate of food, irrespective of the arranged conditions of S+ value. The ratio of food reinforcers on the abscissa was similarly constructed with the higher food-delivery rate in the numerator. The fitted functions represent Equation 1 with obtained mixed-schedule or overall food rates used for R₁ and R₂. In both panels, relative resistance was greater in the component associated with the higher relative primary reinforcement rate. In addition, relative resistance was a positive function of relative primary reinforcement rates. Equation 1 accounted for similar amounts of variance regardless of whether mixed-schedule or overall food rates were used. The slopes of the fits based on both mixed-schedule [F(1,2)  =  18.9, p < .05] and overall food rates [F(1,2)  =  23.4, p < .05] were significantly nonzero. Thus, behavioral momentum theory appears to provide an adequate account of the resistance to change of observing when the analysis is based on obtained rates of primary reinforcement, but not when it is based on the value of S+ deliveries. The finding that Equation 1 did a good job of accounting for relative resistance regardless of whether mixed-schedule or overall food-delivery rates were used is not surprising given the similarity of those reinforcement rates in all of the present experiments.

The present results and our previous findings suggest that parameters of conditioned reinforcement may contribute to response rates but not to resistance to change. Although Shahan et al. (2003) attributed differences in the effects of rates of primary reinforcement associated with an S+ on resistance to change to difference in the conditioned reinforcing value of the S+, the present results do not support this hypothesis. Consistent with the present findings, Shahan and Podlesnik (2005) found that rate of S+ delivery affected observing rates but not resistance to change; resistance to change only differed when there were differences in obtained rates of primary reinforcement.

Given that observing rates were higher with an S+ associated with a greater value, one could conclude that such an S+ was a more effective conditioned reinforcer. This view would be consistent with a large body of evidence that observing responses are maintained by the conditioned reinforcing effects of S+ (see Dinsmoor, 1983, for review). Our previous findings showed that observing rates are higher with higher rates of S+ delivery (Shahan & Podlesnik, 2005) and that the allocation of concurrently available observing responses matches the relative rate of S+ delivery produced by those observing responses (Shahan et al., 2006). Both of these findings are consistent with the notion that S+ functions as a reinforcer.

The reinforcing effects of S+ are somewhat less clear, however, if one accepts the assertion of behavioral momentum theory that response rates and resistance to change are separable aspects of behavior. Specifically, from the perspective of behavioral momentum theory, resistance to change provides a more appropriate measure of the response-strengthening effects of reinforcers than does response rate. Thus, the fact that variations in the value of an S+ affected response rate but not resistance to change suggests that changes in the value of an S+ may affect response rates through a mechanism other than response strengthening (Shahan & Podlesnik, 2005). In other words, S+ deliveries may not function as conditioned reinforcers in the sense of increasing the strength of behavior that produces them. Schuster (1969) reached a similar conclusion because adding putative conditioned reinforcers to one terminal link of a concurrent-chains procedure failed to produce an increase in preference for that terminal link, despite the fact that response rates were higher in the terminal link in which they were available (see Fantino & Romanowich, 2007, for recent review). Schuster suggested that some other mechanism besides a reinforcement-like strengthening effect might be responsible for the effects of the putative conditioned reinforcers on response rates. Davison and Baum (2006) recently reached a similar conclusion based on an analysis of the effects of added food-paired stimuli on preference in concurrent schedules. They suggested that food-associated stimuli have their effects as a result of a signaling function as opposed to a response-strengthening effect. Related information-based reinterpretations of phenomena once attributed to conditioned reinforcement have been offered by a number of learning theorists over the last 40 years (see Williams, 1994a, for review). By showing that the value of an S+ impacts response rates but not response strength as measured by resistance to change, the present findings further contribute to ambiguity about the appropriateness of attributing response-strengthening effects to an S+.

However, Shahan and Podlesnik (2005) noted that before results from resistance-to- change tests are used to dismiss potential response-strengthening effects of an S+, it is important to consider the fact that resistance to change depends on the Pavlovian stimulus–reinforcer relation between a reinforcer and the discriminative context in which it occurs. Thus, even if an S+ does function as a reinforcer, its response-strengthening effects might not be detectable using resistance-to-change tests. The reason is that any transfer of value from S+ to the broader discriminative context would require second-order conditioning (i.e., establishing CS₂ through association with CS₁ rather than a US). If failures of second-order conditioning are common with conditioned reinforcers (CS₁) and discriminative contexts (CS₂), any response-strengthening effects of an S+ could be difficult to detect using resistance-to-change tests. Williams and Dunn (1991) noted that a similar lack of second-order conditioning might be the reason that Schuster (1969) failed to show an increase in preference for a terminal link stimulus (CS₂) associated with additional food-associated stimuli (CS₁). Unfortunately, the role of failures of second-order conditioning in both Schuster's results with concurrent chains, and in our results with resistance to change of observing, remains unclear. Regardless, our results suggest that behavioral momentum theory might not be easily applicable to parameters of conditioned reinforcement because (a) the putative conditioned reinforcers are not reinforcers in the sense of increasing response strength, or (b) failures of second-order conditioning may prevent putative conditioned reinforcers from impacting resistance to change as mediated through the overall stimulus–reinforcer relation.

One finding from the present experiment might seem to suggest that an S+ can impact response strength. In Experiment 2, resistance to extinction was greater when observing responses continued to produce S+ than when they did not. Bell, Seip, and Fitzsimmons (2007) obtained a similar result using a signaled delay to reinforcement procedure. As noted by Williams (1994b) similar resistance to extinction tests with and without a putative conditioned reinforcer were often used in the early research on conditioned reinforcement, but were abandoned because of interpretive issues. The difficulty is that extinction with the putative conditioned reinforcer present is associated with a smaller change in stimulus conditions (i.e., generalization decrement) than extinction without the putative conditioned reinforcer. Thus, differences in resistance to extinction with and without an S+ in Experiment 2 and in Bell et al. provide no unambiguous evidence of response-strengthening effects of an S+.

In conclusion, the present experiments showed that variations in value of an S+ produced variations in observing response rates, but had no effect on resistance to change. One interpretation of these results is that S+ deliveries may not contribute to response strength (i.e., function as conditioned reinforcers) and that the effects of S+ value on response rates are the result of some other mechanism. Our previous finding that variations in rates of S+ delivery affect response rates but not resistance to change is also consistent with this conclusion. However, additional examinations of the effects of parameters of putative conditioned reinforcers on resistance to change with other procedures should be conducted. Finally, additional research will be required to determine whether potential failures of second-order conditioning may play a role in our failures to find effects of parameters of conditioned reinforcement on resistance to change. Such additional research could help to determine if putative conditioned reinforcers contribute to response strength and whether behavioral momentum theory will ultimately be applicable to parameters of conditioned reinforcement.

APPENDIX A

Mixed and S+ stimuli for each pigeon in each experiment.

Experiment	Pigeon	Component stimuli
		Mixed		S+
		Low-value	High-value	Low-value	High-value
1	77	triangle	plus sign	Blue	red
	210	horizontal line	vertical line	Green	yellow
	206	horizontal line	vertical line	Yellow	green
	38	vertical line	horizontal line	Yellow	green
2 & 3(1)	225	red	blue	Turquoise	yellow
	270	blue	red	Yellow	turquoise
	622	blue	red	Yellow	turquoise
	199	red	blue	Turquoise	yellow
3(2)	225	green	white	Circle	vertical line
	270	green	white	Circle	vertical line
	622	white	green	vertical line	circle
	199	white	green	vertical line	circle

APPENDIX B

Response rates and number of baseline (BL) sessions in each condition of Experiment 1. Conditions are presented in the order they occurred. Baselines are indicated by the following disrupter (presession feeding [PF] and food and S+ extinction [E]). Baseline response rates are the mean rates from the final six sessions of baseline prior to disruption. Response rates from individual sessions of disruption are presented. SDs are in italics.

Cond	Subject	Sess	Response Rate
			High Value			Low Value
			Obs	S+	Mixed	Obs	S+	Mixed
PF BL	77	22	28.16	90.59	60.43	28.34	84.50	31.61
			2.54	8.44	8.60	1.91	9.51	6.47
PF		1	20.77	92.04	42.37	28.44	78.59	19.76
PF		1	19.45	88.93	20.35	26.22	80.64	20.23
PF		1	16.55	103.02	39.91	29.92	88.76	30.36
PF		1	19.00	88.47	24.73	22.93	78.11	18.21
PF		1	13.64	65.50	21.15	13.12	73.29	14.04
E BL		32	28.68	85.59	47.89	30.80	73.49	36.68
			3.56	5.92	7.43	4.88	5.75	6.54
E		1	30.95	-	45.05	50.95	-	23.35
E		1	11.20	-	24.05	35.50	-	28.60
E		1	9.85	-	17.20	25.65	-	25.60
E		1	16.85	-	16.40	18.15	-	15.70
E		1	2.00	-	8.85	1.95	-	3.25
PF BL	206	47	42.89	66.63	5.17	11.80	50.06	19.23
			2.83	3.74	1.90	2.17	5.95	2.61
PF		1	40.44	50.15	2.16	10.45	45.97	15.97
PF		1	30.76	40.44	0.50	16.55	41.86	7.35
PF		1	24.92	26.57	1.44	12.98	27.93	7.39
PF		1	14.37	30.02	1.94	6.67	13.95	4.97
PF		1	8.82	26.31	1.22	7.43	23.94	5.30
E BL		14	41.67	79.45	3.28	7.76	61.94	28.25
			2.86	4.88	1.58	1.44	7.09	7.39
E		1	29.85	-	0.05	24.80	-	4.95
E		1	20.90	-	0.00	15.25	-	4.85
E		1	9.10	-	0.00	9.50	-	0.30
E		1	6.45	-	0.00	5.25	-	0.00
E		1	2.05	-	0.00	0.15	-	0.00

APPENDIX C

Response rates and number of baseline (BL) sessions in each condition of Experiment 2. Conditions are presented in the order they occurred. Baselines are indicated by the following disrupter (presession feeding [PF], extinction plus ICI FT food [Eft], ICI FT food [FT], food and S+ extinction [E], and S+ extinction [Es]). Baseline response rates are the mean rates from the final six sessions of baseline prior to disruption. Response rates from individual sessions of disruption are presented. SDs are in italics. Cells with asterisks indicate responding not included in analyses.

Subject	Cond	Sess	Response Rate
			High Value			Low Value
			Obs	S+	Mixed	Obs	S+	Mixed
225	PF BL	95	38.00	57.67	17.52	23.21	64.58	34.76
			3.07	14.24	2.71	2.19	9.73	3.83
	PF	1	12.51	47.76	12.88	15.96	84.75	32.99
	PF	1	23.37	41.37	16.79	17.24	64.94	35.58
	PF	1	18.59	77.60	14.12	12.95	88.86	34.17
	PF	1	19.38	54.95	3.28	15.57	78.76	32.55
	PF	1	17.05	111.00	6.10	13.80	87.85	30.46
	Eft BL	6	37.86	47.90	18.50	21.15	66.36	31.85
			2.57	15.08	2.72	1.65	11.37	3.98
	Eft	1	21.81	-	6.00	14.65	-	18.81
	Eft	1	19.96	-	6.30	12.45	-	11.15
	Eft	1	13.25	-	1.70	5.85	-	5.15
	Eft	1	14.80	-	2.15	13.45	-	4.60
	Eft	1	11.25	-	2.75	15.25	-	11.20
	FT BL	21	38.46	59.55	14.44	21.93	66.48	35.95
			3.37	14.83	2.29	1.69	8.86	2.55
	FT	1	23.95	38.10	6.35	25.96	67.32	22.58
	FT	1	22.36	48.65	5.46	20.19	46.77	21.04
	FT	1	27.73	51.46	5.07	19.14	28.89	20.70
	FT	1	29.75	60.04	7.46	18.56	45.00	21.38
	FT	1	29.45	55.92	6.98	13.96	56.68	24.43
	E BL	36	35.08	53.72	21.51	22.05	61.56	36.21
			2.99	14.57	6.17	2.12	2.73	0.50
	E	1	30.11	-	31.21	21.61	-	38.16
	E	1	28.91	-	23.41	18.76	-	31.01
	E	1	26.11	-	16.20	13.45	-	27.16
	E	1	16.11	-	8.65	7.85	-	8.65
	E	1	6.70	-	4.15	2.55	-	2.40
	E	1	5.25	-	3.30	5.40	-	7.50
	E	1	4.75	-	2.60	4.95	-	7.15
	E	1	7.10	-	2.70	1.65	-	1.70
	E	1	4.75	-	0.45	0.95	-	1.20
	E	1	5.75	-	2.70	2.30	-	2.55
	Es BL	6	27.50	51.98	17.29	18.78	63.60	36.90
			1.47	18.86	4.13	1.52	6.94	6.13
	Es	1	23.90	64.04	11.13	31.46	61.22	20.24
	Es	1	18.94	46.43	6.35	27.80	51.49	10.48
	Es	1	10.94	17.21	2.67	20.50	55.92	7.16
	Es	1	17.22	26.02	4.21	16.14	31.42	7.60
	Es	1	10.43	12.01	1.37	15.94	49.59	6.78
	Es	1	7.12	38.69	0.62	13.76	46.92	8.00
	Es	1	9.28	62.04	2.10	19.04	53.13	14.07
	Es	1	7.90	26.02	1.08	11.76	57.90	7.33
	Es	1	10.00	52.03	2.92	17.31	51.91	8.72
	Es	1	2.08	*	2.00	9.32	*53.90	8.37
	Es	1	1.96	64.04	2.26	7.10	77.63	7.96
	Es	1	2.09	45.36	2.81	1.71	48.03	2.61
	Es	1	8.73	52.03	8.52	13.12	57.75	14.43
	Es	1	2.06	40.03	2.05	4.68	32.34	3.43
	Es	1	2.62	36.02	2.43	2.68	18.92	3.47
	Es	1	1.28	60.04	1.22	1.75	11.11	2.05
	Es	1	0.06	8.01	0.05	0.12	4.54	0.43
	Es	1	20.27	18.68	4.52	12.44	57.01	10.52
	Es	1	24.46	4.00	3.70	11.15	61.26	8.79
	Es	1	9.34	16.01	1.28	7.65	7.00	1.94
622	PF BL	61	27.00	97.73	23.34	23.78	89.55	35.09
			4.40	16.89	6.16	1.85	8.00	4.80
	PF	1	3.44	52.03	8.36	20.75	86.93	25.00
	PF	1	8.88	111.99	35.06	29.27	80.67	40.85
	PF	1	14.94	87.52	36.24	25.07	101.18	41.10
	PF	1	15.95	44.03	29.03	26.57	101.29	40.97
	PF	1	8.82	67.16	30.92	11.36	77.24	63.44
	Eft BL	42	30.62	110.03	23.68	19.46	100.39	51.24
			4.06	18.31	3.00	1.99	6.98	7.11
	Eft	1	24.51	-	5.80	28.11	-	17.01
	Eft	1	19.66	-	3.65	30.76	-	11.95
	Eft	1	16.41	-	2.20	26.11	-	7.95
	Eft	1	9.85	-	2.90	18.16	-	13.10
	Eft	1	7.50	-	2.50	14.15	-	15.65
	FT BL	12	29.36	93.98	26.82	17.60	96.04	65.95
			4.28	23.02	6.82	1.59	3.56	4.41
	FT	1	13.22	81.95	18.98	22.37	78.14	36.41
	FT	1	14.89	79.38	16.57	19.68	54.22	34.47
	FT	1	15.48	91.75	18.18	17.78	59.61	50.01
	FT	1	16.75	67.42	17.98	27.52	70.94	41.25
	FT	1	16.20	98.03	16.30	22.38	50.89	41.18
	E BL	24	25.75	84.29	17.46	24.23	73.90	55.53
			3.53	23.55	7.26	2.61	8.70	8.08
	E	1	26.11	-	19.41	19.56	-	54.42
	E	1	18.76	-	13.25	24.06	-	53.87
	E	1	11.60	-	12.00	24.06	-	48.97
	E	1	10.55	-	10.10	20.01	-	48.36
	E	1	5.85	-	3.95	20.76	-	34.96
	E	1	6.80	-	4.90	21.56	-	24.76
	E	1	11.35	-	7.45	19.41	-	32.76
	E	1	4.85	-	2.55	13.50	-	35.51
	E	1	6.55	-	4.35	15.75	-	35.76
	E	1	8.40	-	2.40	11.90	-	28.11
	Es BL	17	31.75	102.69	13.54	24.42	84.50	55.46
			4.43	15.57	3.71	1.70	7.54	4.13
	Es	1	30.40	105.40	12.94	22.78	96.63	51.87
	Es	1	36.53	56.04	8.21	29.73	85.18	43.12
	Es	1	28.19	73.27	8.42	36.49	103.89	30.69
	Es	1	34.66	96.06	3.79	40.55	96.73	17.63
	Es	1	26.90	92.06	3.95	41.96	89.89	13.11
	Es	1	22.14	68.05	3.07	51.83	76.93	11.16
	Es	1	23.11	92.06	2.58	59.58	87.56	14.57
	Es	1	19.64	62.71	2.81	42.76	56.51	7.14
	Es	1	9.49	48.03	2.26	48.42	84.60	7.81
	Es	1	27.54	61.37	2.03	55.06	87.70	7.56
	Es	1	15.33	86.93	0.93	56.62	81.20	6.17
	Es	1	12.91	107.63	1.08	58.98	74.78	4.21
	Es	1	18.11	68.05	1.44	39.16	72.47	8.07
	Es	1	7.29	8.01	0.91	31.49	59.99	3.16
	Es	1	11.80	77.38	2.44	19.93	46.68	0.68
	Es	1	5.50	*	0.25	12.16	*26.59	0.48
	Es	1	5.27	80.05	0.20	22.70	23.49	0.13
	Es	1	0.05	*	0.00	7.23	*15.39	0.12
	Es	1	0.00	*	0.00	16.07	*21.88	0.25
	Es	1	0.20	*	0.20	20.56	*13.38	0.19

APPENDIX D

Response rates and number of baseline (BL) sessions in each condition of Part 1 of Experiment 3. Conditions are presented in the order they occurred. Baselines are indicated by the following disrupter (presession feeding [PF], ICI FT food [FT], and S+ extinction [Es]). Baseline response rates are the mean rates from the final six sessions of baseline prior to disruption. Response rates from individual sessions of disruption are presented. SDs are in italics. Cells with asterisks indicate responding not included in analyses.

Cond	Subject	Sess	Response Rate
			High Value			Low Value
			Obs	S+	Mixed	Obs	S+	Mixed
PF BL	225	47	25.53	44.29	29.14	18.46	64.92	34.07
			2.73	13.47	2.70	1.95	6.94	3.22
PF		1	14.30	53.52	20.27	15.54	88.12	30.72
PF		1	16.77	78.05	27.85	14.17	96.67	27.50
PF		1	13.48	37.27	26.29	19.73	59.70	30.67
PF		1	11.77	80.53	26.50	11.98	83.54	29.68
PF		1	11.74	52.92	28.30	9.69	78.62	22.85
FT BL		19	25.81	32.04	26.56	20.63	62.28	28.22
			3.13	6.02	2.96	1.45	6.59	3.87
FT		1	18.48	36.87	12.86	17.98	47.55	21.60
FT		1	16.59	32.49	17.58	17.11	38.98	21.70
FT		1	12.34	21.93	18.49	17.57	44.16	22.79
FT		1	14.22	46.93	21.23	15.49	57.44	25.02
FT		1	15.61	56.51	20.69	10.39	56.00	26.48
Es BL		10	28.18	22.36	23.30	22.69	54.86	28.16
			3.76	7.89	4.35	0.88	4.53	4.79
Es		1	20.06	28.69	37.36	24.12	63.38	34.41
Es		1	23.38	32.02	30.17	21.21	64.70	25.53
Es		1	30.96	35.02	22.59	19.60	59.64	22.87
Es		1	24.58	36.02	14.19	13.11	52.98	20.06
Es		1	27.01	10.01	14.06	11.28	50.85	10.09
Es		1	17.01	23.02	8.90	9.56	36.77	12.13
Es		1	12.98	43.09	5.02	1.81	32.02	2.08
Es		1	16.21	37.28	5.58	5.82	42.61	4.81
Es		1	4.73	106.35	1.86	1.03	8.01	0.56
Es		1	9.32	44.03	3.70	3.78	29.35	4.33
Es		1	12.22	61.52	4.78	4.05	80.05	5.28
Es		1	3.35	*	1.10	0.20	*	0.05
Es		1	13.18	44.03	5.44	0.91	71.84	0.97
Es		1	0.46	16.01	0.15	0.58	12.01	0.37
Es		1	2.20	*	0.00	2.31	*	2.51
Es		1	18.16	66.04	5.59	11.09	75.25	10.46
Es		1	12.93	44.03	4.77	2.42	58.04	3.21
Es		1	0.50	*	0.05	1.69	*	0.82
Es		1	1.90	*	0.00	0.00	*	0.00
Es		1	1.20	*	0.15	1.16	*	0.37
PF BL	622	66	24.62	92.53	36.92	19.57	84.09	57.83
			3.19	13.50	6.06	2.19	6.05	6.59
PF		1	10.48	83.97	30.95	18.09	83.57	39.90
PF		1	18.84	65.50	31.58	20.05	94.93	44.73
PF		1	26.49	91.10	34.48	18.93	92.01	47.03
PF		1	29.27	107.66	37.73	19.41	95.63	37.76
PF		1	22.81	77.64	38.73	20.56	107.95	39.22
FT BL		12	22.88	77.68	43.46	17.61	91.64	56.56
			3.46	6.53	3.98	1.90	7.52	2.50
FT		1	24.12	85.78	15.93	27.00	77.52	11.23
FT		1	22.86	70.39	24.60	42.78	71.44	14.14
FT		1	28.06	83.78	16.65	29.96	72.11	29.82
FT		1	26.06	75.58	14.57	27.19	76.53	34.62
FT		1	30.96	96.07	14.21	34.26	71.51	34.40
Es BL		25	25.80	88.60	55.86	21.48	92.74	60.64
			3.77	17.67	6.67	2.62	7.17	5.15
Es		1	24.11	57.64	58.15	24.14	101.07	57.97
Es		1	23.04	74.35	55.25	23.56	95.69	47.84
Es		1	32.12	104.87	37.45	28.18	99.21	30.45
Es		1	39.28	82.72	27.28	32.64	87.84	19.17
Es		1	44.70	84.06	25.80	46.11	92.76	26.25
Es		1	51.32	79.25	27.95	51.85	78.67	33.35
Es		1	43.86	60.04	21.51	45.16	77.58	27.11
Es		1	44.12	71.25	20.22	42.07	86.69	21.95
Es		1	46.68	66.04	23.44	45.06	85.62	23.86
Es		1	39.75	85.39	13.77	39.80	74.19	14.03
Es		1	33.88	88.07	7.49	56.14	65.80	7.70
Es		1	43.85	72.85	11.68	43.05	82.17	10.23
Es		1	36.44	92.06	6.78	43.51	54.86	12.57
Es		1	9.54	78.05	1.59	22.85	50.45	5.48
Es		1	33.03	69.23	10.27	28.55	78.93	9.75
Es		1	5.80	102.07	1.28	6.37	62.19	1.65
Es		1	3.15	*	0.80	16.00	*46.80	9.14
Es		1	1.82	8.01	0.15	2.45	25.62	0.11
Es		1	9.09	96.04	4.91	14.90	39.46	5.84
Es		1	33.08	77.80	10.94	18.44	54.29	4.19

APPENDIX E

Response rates and number of baseline (BL) sessions in each condition of Part 2 of Experiment 3. Conditions are presented in the order they occurred. Baselines are indicated by the following disrupter (presession feeding [PF], ICI FT food [FT], and S+ extinction [Es]). Baseline response rates are the mean rates from the final six sessions of baseline prior to disruption. Response rates from individual sessions of disruption are presented. SDs are in italics. Cells with asterisks indicate responding not included in analyses.

Cond	Subject	Sess	Response Rate
			High Value			Low Value
			Obs	S+	Mixed	Obs	S+	Mixed
PF BL	225	54	36.64	97.18	41.64	23.04	70.13	61.83
			6.73	27.59	7.85	2.88	9.72	3.53
PF		1	14.25	71.93	0.42	14.73	48.74	41.00
PF		1	30.28	52.67	0.86	17.63	62.50	61.13
PF		1	10.23	77.31	0.83	14.88	60.22	46.30
PF		1	1.89	60.04	0.31	3.63	48.90	19.95
PF		1	25.05	62.15	0.65	14.43	61.04	41.70
FT BL		7	45.21	84.43	34.71	20.49	66.75	66.52
			8.36	6.15	8.05	3.48	8.52	7.38
FT		1	13.19	97.11	20.49	28.21	45.42	27.64
FT		1	15.97	62.65	21.12	21.99	27.18	27.87
FT		1	11.48	55.32	21.30	11.00	20.91	19.68
FT		1	9.44	70.13	20.18	12.87	31.21	32.80
FT		1	14.75	61.69	13.95	19.59	31.58	25.16
Es BL		19	38.61	129.59	31.15	22.39	62.04	64.00
			5.78	19.33	5.76	2.79	10.53	6.63
Es		1	47.17	84.06	22.32	18.14	59.66	67.36
Es		1	45.29	106.47	22.51	29.66	59.85	49.61
Es		1	47.29	135.47	23.09	30.36	58.74	40.57
Es		1	56.60	87.06	17.22	24.81	33.24	38.25
Es		1	36.53	40.03	12.58	19.69	29.07	29.61
Es		1	18.62	74.05	6.98	13.00	23.82	18.60
Es		1	53.00	73.38	16.11	20.94	27.76	36.47
Es		1	25.80	43.03	10.69	14.99	20.25	16.32
Es		1	8.90	*	1.00	11.88	*24.32	3.58
Es		1	16.67	88.06	5.18	7.66	29.62	4.69
Es		1	13.46	41.36	4.16	4.38	26.94	2.14
Es		1	14.30	20.81	4.05	4.55	17.73	0.93
Es		1	17.32	40.03	1.67	2.27	8.01	0.27
Es		1	13.32	60.04	3.26	2.13	16.67	0.38
Es		1	2.67	14.01	0.77	6.23	11.11	1.35
Es		1	18.86	*	0.65	3.94	*3.63	0.17
Es		1	5.39	60.04	0.92	3.83	10.41	0.34
Es		1	4.20	*	0.00	0.74	*2.00	0.00
Es		1	24.47	41.36	3.12	7.53	12.68	1.00
Es		1	5.85	0.00	0.15	1.49	1.60	0.00
PF BL	622	101	18.68	46.89	59.59	22.26	99.67	63.34
			1.07	7.22	8.57	1.85	7.87	5.31
PF		1	13.53	21.07	50.26	12.86	67.56	45.07
PF		1	5.32	20.70	29.21	14.92	69.89	26.89
PF		1	8.19	43.36	52.73	20.51	77.08	45.32
PF		1	13.40	57.94	43.32	16.17	84.16	29.90
PF		1	13.39	59.24	49.80	17.73	90.96	41.22
FT BL		11	12.26	60.01	57.20	16.20	98.76	67.62
			0.93	16.40	7.11	2.02	5.98	5.58
FT		1	17.38	54.46	19.87	18.53	56.96	33.15
FT		1	15.25	61.46	22.79	16.98	44.25	28.12
FT		1	10.37	39.47	22.07	7.35	55.85	30.65
FT		1	9.23	49.69	25.47	9.28	48.08	26.00
FT		1	8.39	54.04	35.58	11.66	56.05	26.56
Es BL		13	14.70	48.44	47.49	15.04	85.99	60.35
			2.07	17.53	3.73	2.57	5.79	4.53
Es		1	11.54	42.70	50.46	21.79	90.67	48.97
Es		1	16.27	54.04	47.33	27.32	77.81	44.01
Es		1	20.95	51.03	35.91	26.30	72.59	35.90
Es		1	26.38	29.02	32.06	43.56	84.50	1.95
Es		1	20.79	40.03	31.39	26.27	82.86	35.21
Es		1	19.49	48.03	24.21	22.39	69.80	29.46
Es		1	16.94	52.03	21.93	20.68	49.81	22.93
Es		1	18.06	62.04	16.95	18.21	44.39	8.42
Es		1	18.01	76.05	10.60	22.61	30.45	24.48
Es		1	14.98	88.06	6.82	26.00	48.85	5.04
Es		1	13.82	63.12	14.08	18.65	48.33	17.09
Es		1	15.10	43.23	20.06	21.23	36.39	11.48
Es		1	13.04	93.40	11.38	16.98	27.25	5.11
Es		1	11.06	51.03	6.05	12.01	20.48	0.44
Es		1	11.43	40.03	7.74	10.79	23.33	3.75
Es		1	12.27	34.42	2.56	15.32	12.01	1.02
Es		1	5.95	40.03	1.05	9.62	21.55	6.97
Es		1	6.63	28.02	2.74	5.79	4.07	3.61
Es		1	9.33	29.19	3.09	7.66	0.55	0.43
Es		1	8.84	4.00	2.79	11.47	6.00	4.80