Matching and Conditioned Reinforcement Rate

Abstract

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.

Considerable empirical and theoretical attention has been directed to the effects of conditioned reinforcement on choice (see Fantino, 1977; Grace, 1994, for reviews). These efforts have focused on data obtained using the concurrent-chains procedure. In the usual procedure, concurrent variable-interval (VI) schedules (conc VI VI) in initial links arrange transitions to terminal links signaled by different stimuli and associated with independent schedules of primary reinforcement. The stimuli associated with the terminal links typically are considered to function as conditioned reinforcers. Thus, the allocation of responding to the initial-link schedules is used as a measure of the effects of the conditioned reinforcing value of the terminal-link stimuli. Despite widespread use of concurrent-chains procedures to study conditioned reinforcement, preference in the initial links cannot be interpreted unambiguously in terms of the conditioned reinforcing properties of the terminal-link stimuli. The ambiguity results in part from the dependency between responding in the initial links and access to the primary reinforcer in the terminal links (cf. Branch, 1983; Dinsmoor, 1983; Williams, 1994).

Concurrent-chains procedures are especially problematic for examining the effects of rate of conditioned reinforcement on choice. Variations in the rate of transition into the terminal links (i.e., conditioned reinforcement rate) produce concomitant changes in the rate of primary reinforcement. This interdependency between rates of conditioned and primary reinforcement is notably reflected in delay reduction theory (Squires & Fantino, 1971). The theory includes no role for rates of conditioned reinforcement, but suggests that the effects of variations in the rate of transition into the terminal links result from changes in the rate of primary reinforcement and the value of the terminal link stimuli (i.e., signaled reduction in delay to primary reinforcement).

Other theories of choice based on extensions of the concatenated generalized matching law to concurrent-chains performance (e.g., the contextual choice model and the hyperbolic value-added model) include roles for both conditioned reinforcement rate and conditioned reinforcement value (e.g., Grace, 1994; Mazur, 2001). The concatenated generalized matching law (Baum, 1974; Baum & Rachlin, 1969) suggests that the ratio of responses to two alternatives is a power function of the relative value of the reinforcers provided by those alternatives. Ignoring for present purposes the effects of other parameters of reinforcement (e.g., immediacy), the values of the alternatives may be considered to result from the multiplicative effects of relative reinforcement rate and magnitude. Thus:

1

where B is response rates, R is reinforcement rates, A is reinforcement magnitude, and subscripts refer to the two alternatives. The parameters a1 and a2 represent sensitivity of response ratios to variations in reinforcement rate and magnitude, respectively. The parameter b represents systematic bias for one alternative that is unrelated to variations in reinforcement parameters. Extensions of the concatenated matching law to concurrent-chains performance are based on Davison's (1983) suggestion that initial-link response allocation can be modeled by assuming that R₁ and R₂ represent the rates of transition into the terminal links. Terms representing the value of the terminal-link stimuli then can be substituted for reinforcement magnitude (i.e., A). When there are no terminal links, standard concurrent schedules of reinforcement are in effect. These models then reduce to the generalized matching law relating behavior ratios to ratios of primary reinforcement. Although they differ in how they deal with terminal-link value, the contextual choice model and the hyperbolic value-added model share Davison's assumption that rates of primary and conditioned reinforcement (i.e., terminal-link transitions) have functionally equivalent effects on choice. Evaluating this assumption is difficult with concurrent-chains procedures because of the direct effects of changes in the durations of the initial links on primary reinforcement rate.

Limited evidence of a role for conditioned reinforcement rate in governing choice in the absence of changes in primary reinforcement rate has come from modified concurrent-chains procedures (Mazur, 1999; Williams & Dunn, 1991). In these procedures, additional terminal-link entries are sometimes produced by responses in the initial links. The added terminal-link entries are associated with extinction; therefore, the rate of primary reinforcement remains unchanged as the relative rate of the added terminal-link entries is varied. Both Williams and Dunn (1991) and Mazur (1999) found small increases in preference for an initial-link schedule associated with a four-fold higher rate of terminal-link entry. Unfortunately, interpretation of these results is complicated by the fact that Mazur also found that this effect only occurred when the same stimulus was used for both terminal links. When different stimuli were used for the terminal-link stimuli, there was no difference in initial-link preference or a preference for the initial link providing the lower rate of terminal-link entry (cf. Dunn, Williams, & Royalty, 1987). Furthermore, Mazur showed that preference for the initial link associated with the higher rate of terminal-link entry tended to decrease with exposure to the procedure. Mazur suggested that the small and transitory effects of conditioned reinforcement rate on choice in these procedures occur only when subjects have difficulty discriminating the source of the terminal-link transitions and before they learn that the added stimulus presentations are not associated with food. As the subjects learn that the added stimulus presentations are not associated with food, the value of the added stimuli decreases. Thus, a thorough assessment of the functional similarity of the effects of primary and conditioned reinforcement on choice will require variations in relative rate of conditioned reinforcement in the absence of changes in primary reinforcement rate and in changes in the value of the conditioned reinforcer.

Shahan and Podlesnik (2005) recently found that variations in the rate of conditioned reinforcement affected the rate of observing responses in the absence of changes in the rate of primary reinforcement. In their procedure, a multiple schedule of observing-response procedures was arranged. Each component of the multiple schedule included alternating but unsignaled periods of access to a random-interval (RI) schedule and extinction (i.e., a mixed schedule). Responses on an observing key produced 15-s of exposure to stimuli differentially associated with the RI (S+) and extinction (S−) periods. The obtained rate of primary reinforcement delivered by the RI schedule in the two components was the same. Observing responses produced stimulus presentations at a high rate (e.g., on an RI 15-s schedule) in one component and at a lower rate (e.g., on an RI 60-s schedule) in the other component. Observing rates were higher in the component associated with more frequent stimulus presentations. Based on the assumption that the observing responses were maintained by S+ deliveries, these results suggest that higher rates of conditioned reinforcement maintain higher response rates, even in the absence of differences in rates of primary reinforcement.

In addition, with the observing-response procedure, the value of S+ should remain constant with variations in the rate of S+ delivery. This constancy occurs because the ratio of the rate of primary reinforcement delivered in S+ to the rate of primary reinforcement delivered in the mixed schedule should remain roughly constant with variations in the rate of S+ delivery. With the exception of the extreme case in which all primary reinforcers are obtained during S+, this ratio remains constant because both primary reinforcers and time are redistributed from the mixed schedule to S+ as the rate of S+ delivery increases—thus rates of primary reinforcement in both the mixed schedule and during S+ remain roughly constant. Similar constancy is obtained when conditioned reinforcing value is considered in terms of the relative reduction in delay to primary reinforcement signaled by the onset of S+. Thus, the results of Shahan and Podlesnik (2005) suggest that variations in the rate of conditioned reinforcement affect response rates in the absence of changes in both the rate of primary reinforcement and the value of the conditioned reinforcer. However, Shahan and Podlesnik did not examine the effects of rate of conditioned reinforcement on choice.

A procedure that should allow examination of the effects of variations in the relative rate of conditioned reinforcement on choice is the concurrent observing-response procedure (e.g., Dinsmoor, Mulvaney, & Jwaideh, 1981). In this procedure, two concurrently available observing responses produce a stimulus associated with access to a VI schedule of primary reinforcement (S+) in a context in which alternating periods of the VI schedule and extinction on a third key are otherwise unsignaled (i.e., mixed schedule). This procedure shares some important characteristics with the concurrent-chains procedure, but it differs in term of how variations in rate of conditioned reinforcement affect rates of primary reinforcement. For example, responding to the concurrently available observing keys in the mixed schedule can be considered analogous to responding in the initial links of concurrent-chains schedules. Similarly, the transition between the mixed-schedule stimuli and S+ is analogous to the transition between the initial and terminal links. The important difference is that with concurrent-chains schedules variations in the rate of terminal-link entry produced by an option also change the rate of food produced by responding to that option. This change in reinforcement rate occurs because the same response typically produces both the conditioned (i.e., terminal-link entry) and primary reinforcers, and access to the primary reinforcer in a terminal link requires completion of the associated initial-link schedule. When one initial link is shorter than the other, the rate of primary reinforcement produced by responding to that initial link is higher. In contrast, in the concurrent observing-response procedure the schedules of primary reinforcement in the mixed schedule operate completely independently of responding on the observing keys. There is no relation between the rate of S+ delivery produced by an observing response and primary reinforcement rate because observing responses have no effect on the availability of the primary reinforcer. All primary reinforcement deliveries may be earned in the mixed schedule in the absence of any observing responses.

The present experiment examined the effects of variations in the relative rate of conditioned reinforcement on choice using a concurrent observing-response procedure. In order to permit a generalized matching law analysis of the effects of variations in the rate of conditioned reinforcement on choice, the relative rate of S+ deliveries produced by the observing responses was varied across conditions. As in the multiple-schedule study of Shahan and Podlesnik (2005), the rate of primary reinforcement and the expected value of the conditioned reinforcers remained constant across variations in relative conditioned reinforcement rate.

Method

Subjects

The subjects were 4 homing pigeons maintained at approximately 80% of their free-feeding weights (+/− 15 g) by postsession supplemental feeding as necessary. The pigeons varied in age and had extensive experience with procedures similar to those arranged in the present experiment. When not in the experimental sessions, the pigeons were housed in individual cages in a temperature-controlled colony with a 12:12 hr light/dark cycle (lights on at 7:00 a.m.) and had free access to water.

Apparatus

The experiment was conducted in four Lehigh Valley Electronics pigeon chambers measuring 350 mm long, 350 mm high, and 300 mm wide. Three response keys were centered on the front panel 83 mm apart (center to center) and were 240 mm above the floor. The keys measured 25 mm in diameter, required about 0.1 N to operate, and could be transilluminated white or green. Each response to an active key produced a 0.01-s flash of the houselight that served as response feedback. Primary reinforcers consisted of 2-s presentations of Purina pigeon checkers from a hopper. The hopper was accessible, when raised, through a 50-mm wide by 55-mm tall aperture located on the midline of the work panel with its center 100 mm from the floor. A 28-V DC clear bulb illuminated the aperture, and all other lights were extinguished when the hopper was operated. General illumination was provided by a shielded 28-V DC clear bulb mounted 45 mm above the center key. A ventilation fan and white noise masked extraneous sounds. Control of experimental events and data recording were conducted with Med Associates® programming and interfacing.

Procedure

As a result of the pigeons' exposure to similar procedures in the past, they were placed immediately on the final procedure. A concurrent observing-response procedure in which observing responses produced only S+ was used (cf. Dinsmoor et al., 1981). Sessions began with the presentation of the nondifferential mixed-schedule stimulus (i.e., white) on each of the three response keys. In the presence of the mixed-schedule stimulus, a VI 90-s schedule of food delivery alternated with extinction on the center key (i.e., food key). The VI and extinction periods were unsignaled and alternated on a separate arithmetic variable-time 90-s schedule with six durations selected randomly from a list ranging from 15 s to 165 s. In the absence of observing responses, the mixed schedule remained in effect. Observing responses to the right or left key could change the three keys to green (S+) for 15 s (exclusive of food hopper time) if the VI 90 s of food delivery was in effect on the food key and an S+ delivery had been arranged for that observing response. Observing responses occurring when extinction was in effect on the food key or during S+ deliveries had no effect. If the schedule on the food key changed from VI to extinction during an S+ delivery, S+ was replaced with the mixed-schedule stimulus. Thus, the program could be in one of three states: mixed schedule stimuli present with the VI schedule in effect on the food key, mixed schedule stimuli present with extinction in effect on the food key, and S+ present with the VI schedule in effect on the food key.

The production of stimulus presentations by the two observing responses was arranged by conc VI VI schedules. The VI schedules for the left and right observing responses arranged ratios of S+ delivery of 1∶9 (VI 100 s, VI 11.1 s), 1∶3 (VI 40 s, VI 13.3 s), 1∶1 (VI 20 s, VI 20 s), 3∶1 (VI 13.3 s, VI 40 s), and 9∶1 (VI 11.1 s, VI 100 s) across conditions. The overall programmed rate of S+ delivery was held constant at 6 per min of time spent in the VI schedule on the food key. Pigeons were exposed to each of the S+ delivery ratios across conditions with a replication of one ratio value. Conditions remained in effect for at least 15 sessions and until relative and absolute observing-response rates (observing responses during mixed schedule/time in mixed schedule) appeared stable for at least six sessions as judged visually. Table 1 shows the order of conditions and the number of sessions in each condition. Table 1 also shows that the overall rate of food delivery (i.e., “foods”) remained near the expected 0.33 foods/min (≈0.5 of the session in extinction and 0.5 in VI 90 s) across conditions for all pigeons.

Table 1. Order of conditions, number of sessions per condition, overall food deliveries per min on center key, total time (min) spent in S+, S+ deliveries per min for left and right observing keys and their ratio, and S+ deliveries per min excluding time spent in extinction during the mixed schedule. All data are means (SD) from the final six sessions of each condition.

Subject	Cond	Sess	Overall food rates	Time in S+	S+ delivery rates
					Overall			w/o Extinction Time
					Left	Right	Ratio (L/R)	Left	Right
225	1∶1	34	0.30	14.03	0.64	0.66	0.97	1.78	1.87
			(0.05)	(2.43)	(0.20)	(0.12)		(0.23)	(0.17)
	1∶3	36	0.32	13.34	0.31	0.94	0.33	0.84	2.54
			(0.05)	(1.13)	(0.05)	(0.09)		(0.12)	(0.15)
	1∶9	41	0.31	13.23	0.14	1.10	0.13	0.41	3.12
			(0.02)	(1.56)	(0.02)	(0.16)		(0.06)	(0.28)
	3∶1	47	0.34	13.57	0.94	0.34	2.79	2.49	0.90
			(0.05)	(2.27)	(0.21)	(0.05)		(0.37)	(0.05)
	9∶1	32	0.33	13.57	1.22	0.14	8.81	3.39	0.39
			(0.05)	(1.73)	(0.17)	(0.03)		(0.12)	(0.08)
	1∶9	33	0.33	13.32	0.17	1.12	0.15	0.45	2.97
			(0.06)	(2.09)	(0.04)	(0.21)		(0.06)	(0.15)
270	1∶1	52	0.32	11.21	0.46	0.53	0.88	1.18	1.33
			(0.04)	(0.51)	(0.06)	(0.08)		(0.22)	(0.09)
	3∶1	26	0.32	10.65	0.66	0.26	2.55	1.64	0.64
			(0.01)	(0.94)	(0.09)	(0.03)		(0.24)	(0.06)
	9∶1	35	0.32	8.63	0.62	0.13	4.73	1.51	0.31
			(0.07)	(2.00)	(0.13)	(0.04)		(0.28)	(0.08)
	1∶3	51	0.34	9.22	0.26	0.53	0.49	0.57	1.16
			(0.03)	(1.12)	(0.07)	(0.07)		(0.17)	(0.12)
	1∶9	46	0.34	11.07	0.11	0.86	0.13	0.26	2.01
			(0.03)	(1.32)	(0.02)	(0.14)		(0.06)	(0.29)
	9∶1	36	0.31	8.09	0.53	0.14	3.87	1.24	0.32
			(0.05)	(1.07)	(0.10)	(0.03)		(0.24)	(0.05)
622	1∶1	69	0.34	14.88	0.66	0.76	0.87	1.84	2.11
			(0.04)	(1.10)	(0.09)	(0.08)		(0.22)	(0.11)
	1∶3	29	0.31	15.08	0.29	1.18	0.25	0.86	3.45
			(0.05)	(1.51)	(0.06)	(0.14)		(0.12)	(0.10)
	1∶9	43	0.33	15.40	0.12	1.38	0.08	0.32	3.82
			(0.05)	(1.77)	(0.04)	(0.20)		(0.08)	(0.13)
	1∶1	50	0.32	14.02	0.65	0.70	0.93	1.91	2.04
			(0.03)	(1.40)	(0.06)	(0.11)		(0.12)	(0.13)
	3∶1	20	0.31	13.33	0.90	0.35	2.57	2.60	1.01
			(0.02)	(1.14)	(0.08)	(0.05)		(0.15)	(0.12)
	9∶1	63	0.33	13.64	1.15	0.15	7.42	3.32	0.45
			(0.03)	(0.47)	(0.06)	(0.03)		(0.20)	(0.06)
199	1∶1	47	0.35	11.83	0.36	0.71	0.51	0.84	1.64
			(0.04)	(0.98)	(0.04)	(0.11)		(0.10)	(0.17)
	3∶1	40	0.31	11.05	0.73	0.27	2.74	1.95	0.71
			(0.02)	(1.03)	(0.07)	(0.04)		(0.18)	(0.09)
	9∶1	45	0.31	9.72	0.73	0.12	5.97	1.86	0.31
			(0.04)	(1.35)	(0.11)	(0.03)		(0.32)	(0.06)
	1∶3	18	0.28	10.33	0.32	0.61	0.52	0.91	1.75
			(0.03)	(0.66)	(0.04)	(0.04)		(0.12)	(0.13)
	1∶9	15	0.28	10.09	0.14	0.76	0.18	0.40	2.13
			(0.06)	(1.46)	(0.01)	(0.12)		(0.06)	(0.13)
	1∶1	18	0.31	10.83	0.49	0.48	1.01	1.30	1.28
			(0.03)	(0.66)	(0.06)	(0.09)		(0.14)	(0.24)

The VI schedules for the two observing responses stopped timing during extinction periods on the food key and during S+ deliveries produced by either observing response. An S+ delivery scheduled for an observing response but undelivered before a transition to extinction on the food key was held until the VI schedule of food delivery was again available on the food key. A 3-s changeover delay was arranged for all switches between keys. Thus, neither food nor S+ could be delivered for a response on a key if a response on any other key had occurred in the previous 3 s. The first response following the 3-s changeover delay could produce an arranged food or S+ delivery. The VI schedules of food and S+ delivery were composed of 10 intervals generated using the method described by Fleshler and Hoffman (1962). Time during food-hopper presentations was excluded from the timing of all events. Sessions were 60 min long and usually were conducted 7 days per week. Raw data summed across the last six sessions of each condition are available in the Appendix.

Results

Figure 1 shows response rates on the left and right observing keys as a function of the programmed ratio of S+ delivery rates. In all cases, observing rates were higher on the key providing the higher rate of S+ delivery. The difference in response rates on the two observing keys also tended to be larger at more extreme ratios of S+ delivery rates. Overall observing rates were higher for Pigeons 225 and 622 than for Pigeons 270 and 199. Table 1 shows total time spent in the presence of S+ as a result of observing responses. Across pigeons, the time spent in the presence of S+ did not vary systematically with variations in the relative rate of S+ delivery.

Fig 1. Mean response rates on the left and right observing keys in the last six days of each condition as a function of the ratio of S+ deliveries provided by those responses.

Data points not connected by the lines are from the replicated condition. Error bars represent ± 1 SD. Both axes are logarithmic.

The obtained rates of S+ delivery (S+ deliveries/mixed-schedule time) for the left and right keys are presented in Table 1. The overall obtained rates of S+ delivery are lower than would be expected based on the overall average of six per min nominally arranged by the VI schedules. These obtained S+ delivery rates partially result from the fact that the VI timers for S+ delivery did not run when extinction was in effect on the food key. Of the roughly 30 min in extinction and 30 min in VI available to be spent in the mixed schedule during each session, the pigeons spent an average of 12 min of the VI time in S+. Thus, on average, 63% of the mixed-schedule time was spent in extinction. As shown in Table 1, excluding the time spent in extinction during the mixed schedule from the calculation of S+ delivery rates brings these rates closer to the expected values. Nonetheless, obtained S+ delivery rates are lower than the arranged values. The remaining difference between the obtained and the arranged S+ delivery rates results from the relatively high rates of S+ arranged by the VI schedules, the fact that the pigeons were responding on three concurrently available schedules of reinforcement (i.e., left observing, right observing, and the food key) during the mixed schedule, and the 3-s changeover delay for all switches. Although there was a tendency for obtained ratios of overall S+ delivery rates to be less extreme than programmed, the obtained ratios increased with increases in the programmed ratios and provided a wide range of values.

Figure 2 shows the relation between the log ratio of observing responses on the left and right keys and the log ratio of obtained S+ deliveries. The data were analyzed using the logarithmic version of the generalized matching law:

2

where B₁ and B₂ are the two observing responses, and R₁ and R₂ are S+ deliveries provided by the two observing responses. The parameters a and log b reflect sensitivity and bias, respectively. Across pigeons, sensitivity ranged from 0.52 to 0.84, and bias was marginal. Equation 2 fitted the data well, accounting for between 91% and 97% of the variance. The pigeons showing the lowest sensitivity values (i.e., 270 and 199) were the pigeons that also had the lowest overall observing rates and for which the ratios of obtained S+ deliveries were in some cases less extreme than the arranged ratios.

Fig 2. Log observing-response ratios as a function of log obtained S+ delivery ratios.

Data are means from the last six sessions of each condition. Regressions of Equation 2 are shown with slope (a), intercept (b), and variance accounted for (r²).

Table 2 presents response rates on the food key in the mixed schedule and during S+ deliveries produced by the left and the right observing responses. Food-key response rates were always higher during S+ deliveries than during the mixed schedule. Food-key response rates were similar during S+ deliveries produced by the left and right observing responses and did not vary systematically across pigeons with variations in the ratio of S+ delivery rates. The similar food-key response rates during S+ deliveries for the two observing responses provide support for the notion that the value of the S+ deliveries from the two observing responses remained similar across variations in relative S+ delivery rate. If S+ deliveries produced by one of the observing responses were of greater value, then that stimulus might be expected to occasion higher response rates on the food key in its presence.

Table 2. Responses per min on the food key during the mixed schedule and during S+ deliveries produced by the left and right observing keys. Data are means (SD) from the last six sessions in each condition.

Subject	Cond	Response rates on center (food) key
		Mixed	S+ from left	S+ from right
225	1∶1	25.67	45.34	46.96
		(3.36)	(2.87)	(4.17)
	1∶3	50.84	54.68	56.44
		(5.71)	(3.61)	(2.88)
	1∶9	47.70	58.60	59.66
		(4.63)	(2.80)	(1.50)
	3∶1	53.26	84.61	83.78
		(2.28)	(5.86)	(3.52)
	9∶1	28.27	86.90	85.63
		(5.48)	(3.60)	(6.67)
	1∶9	35.04	83.18	79.15
		(4.23)	(8.56)	(4.24)
270	1∶1	79.16	122.02	129.40
		(7.54)	(4.02)	(2.93)
	3∶1	62.68	94.13	102.08
		(8.94)	(5.68)	(9.74)
	9∶1	70.80	80.44	91.45
		(6.14)	(6.76)	(11.47)
	1∶3	72.07	73.52	77.35
		(12.01)	(9.16)	(8.64)
	1∶9	71.21	81.62	80.96
		(11.45)	(4.40)	(1.71)
	9∶1	82.59	88.37	87.85
		(6.63)	(7.01)	(9.99)
622	1∶1	45.25	72.48	70.80
		(3.09)	(2.12)	(4.36)
	1∶3	41.66	73.67	74.53
		(3.73)	(5.00)	(1.33)
	1∶9	47.34	87.05	90.20
		(3.45)	(5.13)	(7.29)
	1∶1	53.38	98.07	104.85
		(1.54)	(3.52)	(1.60)
	3∶;1	57.50	85.90	94.87
		(5.66)	(5.76)	(4.93)
	9∶1	60.71	72.34	75.79
		(5.68)	(6.01)	(9.75)
199	1∶1	48.78	111.08	106.15
		(4.70)	(3.08)	(6.01)
	3∶1	45.49	101.02	102.99
		(3.93)	(5.11)	(8.34)
	9∶1	41.49	94.19	89.72
		(4.95)	(8.09)	(10.42)
	1∶3	41.79	104.10	99.01
		(2.72)	(5.35)	(3.66)
	1∶9	52.06	120.97	107.23
		(3.83)	(6.02)	(3.00)
	1∶1	47.99	98.86	99.33
		(5.84)	(3.17)	(2.43)

Further support for the constancy of value of S+ deliveries is provided by consideration of the obtained rates of food delivery during S+ relative to the mixed schedule. Figure 3 shows that food delivery rates during S+ deliveries produced by the left and right observing responses did not vary systematically across conditions. In addition, food delivery rates during the mixed schedule always were lower than during S+ deliveries and did not vary systematically across conditions. One feature of these data requiring comment is that food delivery rates were lower than expected (i.e., 0.33/min) during the mixed schedule and sometimes higher than expected (i.e., 0.67/min) during S+ deliveries from each of the observing keys. These obtained reinforcement rates result from two sources. First, as noted above, a majority of the mixed schedule was spent in extinction. Thus, the rate of food delivery during the mixed schedule is a weighted average of the zero-food rate for time spent in extinction and the food rate for time spent in the VI 90-s schedule (i.e., 0.67 foods/min). The average rate of food delivery in the mixed schedule when calculated based only on VI time is 0.50 foods/min. Second, during the mixed schedule, the pigeons allocated much of their behavior to the two observing keys, and food deliveries set up during the mixed schedule were not necessarily immediately earned by a food-key response. However, when an S+ delivery was produced by an observing response, the pigeons responded at a high rate on the food key and stopped responding on the observing keys. As a result, much of the time required to earn some food deliveries could be spent in the presence of the mixed schedule, but the food delivery itself might occur during an S+ delivery. The partial dissociation of the distribution of time and food deliveries decreased food rates somewhat during the mixed schedule and increased food rates during S+. Even with the most extreme deviations from expected obtained food rates, the difference between obtained and expected rates represents the allocation of only 2–3 foods per session to S+ when much of the time responding for those reinforcers was spent in the mixed schedule. The converse—the allocation of a reinforcer arranged in S+ to the mixed schedule—was less likely because the pigeons did little else but peck the food key during S+ and were likely to earn those reinforcers during S+. Despite the minor deviations from expected food rates during S+ and the mixed schedule, the ratio of food delivery rates during S+ presentations to rates during the mixed schedule did not differ systematically across conditions for S+ deliveries from the two observing responses. Based on the relative constancy of the ratio of food delivery rates for the two sources of S+ (also shown in Figure 3), the value of S+ for the two observing responses likely did not vary systematically across conditions.

Fig 3. Foods per min delivered during the mixed schedule (lowest line) and during S+ deliveries produced by the left and right observing responses (middle lines) as a function of the ratio of S+ deliveries provided by the two observing responses.

In addition, the ratio of food delivery rates during S+ presentations to rates during the mixed schedule are presented for S+ deliveries produced by the left and right observing responses (upper lines). Data are means from the last six sessions of each condition. Error bars represent ± 1 SD. Both axes are logarithmic.

Discussion

The relation between the relative distribution of observing responses and the relative distribution of S+ deliveries was well described by the generalized matching law. The effects of the relative rate of S+ delivery on choice were obtained in the absence of variations in the rate of primary reinforcement. In addition, because the ratio of primary reinforcement rates in S+ to primary reinforcement rates in the mixed schedule remained roughly constant with variations in the relative rate of S+ deliveries, the value of S+ likely remained constant across conditions. To the extent that S+ deliveries can be considered to function as conditioned reinforcers, the present experiment shows that choice is sensitive to the relative rate of conditioned reinforcement in the absence of changes in the rate of primary reinforcement and changes in the value of the conditioned reinforcer.

Previous experiments using modified concurrent-chains procedures have provided some evidence for a role for conditioned reinforcement rate in governing choice in the absence of changes in primary reinforcement rate (e.g., Mazur, 1999; Williams & Dunn, 1991). Nonetheless, the effects of relative conditioned reinforcement rate on choice in those studies were small, and in the case of Mazur's experiment, transient. The transient nature of the effects probably was related to the fact that variations in relative conditioned reinforcement rate were obtained by delivering additional terminal-link entries associated with extinction. Thus, the value of the stimulus presentations was likely reduced. Finally, these previous studies examined only a limited range of relative conditioned reinforcement rates and did not provide sufficient data to derive meaningful sensitivity estimates. The present experiment avoided changes in value of the conditioned reinforcer while examining a sufficient number of relative rates of conditioned reinforcement to provide sensitivity estimates.

The present sensitivity values are comparable to those obtained in previous experiments examining the effects of variations in relative terminal-link entry rates in concurrent-chains schedules. For example, Davison (1983) found that average sensitivity to terminal-link entry ratios obtained across various delays to reinforcement in the terminal links was 0.66. In a similar procedure, Berg and Grace (2004) obtained an average sensitivity value of 0.72, but this estimate was based on only three terminal-link entry ratios. The mean sensitivity value obtained in the present experiment was 0.66. Thus, estimates of sensitivity to relative conditioned reinforcement rate appear roughly similar despite differences in the relation between responding and access to the primary reinforcer in concurrent-chains and observing-response procedures. In both procedures, sensitivity to relative rate of conditioned reinforcement appears somewhat lower than, but within the normal range of, values obtained with variations in relative rate of primary reinforcement in standard concurrent schedules (Baum, 1979).

The present results are consistent with the assumption that primary and conditioned reinforcers have functionally similar effects on choice. Thus, these data are consistent with general choice theories that assume this functional similarity (e.g., the contextual choice model and the hyperbolic value-added model). At the same time, these results are inconsistent with the characterization of choice provided by delay reduction theory (e.g., Squires & Fantino, 1971). As noted above, delay reduction theory does not include a role for rates of conditioned reinforcement independent of the effects of rates of primary reinforcement. The present experiment produced measures of sensitivity to relative rate of conditioned reinforcement comparable to those obtained in concurrent-chains procedures but in the absence of changes in primary reinforcement rate.

Data from a previous experiment by Dinsmoor et al. (1981) provide the opportunity to examine the sensitivity of concurrent observing responses to relative magnitude, as opposed to rate, of conditioned reinforcement. The procedure used by Dinsmoor et al. was the same as used in the present experiment, except that a VI 80-s schedule alternated with extinction on the food key. The schedules of S+ delivery for the two observing keys remained fixed at VI 30 s, and the relative duration (i.e., magnitude) of S+ deliveries was varied across conditions. For one squad of pigeons, the duration of S+ for one of the keys was always 27 s, and the duration for the opposing key varied from 3 to 81 s. For a second squad of pigeons, S+ on the opposing key ranged from 1 to 27 s. Figure 4 shows a generalized matching analysis of the Dinsmoor et al. data. For this analysis, Equation 2 was fitted to the data with the ratios of programmed durations of S+ substituted for relative S+ delivery rate. Sensitivity of the ratio of observing responses to variations in the ratio of S+ durations was 0.37. This value is considerably lower than the sensitivity values obtained with variations in the relative rate of S+ deliveries in the present experiment. The lower sensitivity obtained with relative magnitude of conditioned reinforcement than with relative rate of conditioned reinforcement is consistent with similar findings with variations in relative magnitude and rate of primary reinforcement (see Davison & McCarthy, 1988, for review). However, a procedural artifact may have contributed to the low sensitivity value obtained in the Dinsmoor et al. experiment. As in the present procedure, observing responses produced only S+, and a transition from the VI schedule to extinction on the food key resulted in the termination of an S+ delivery. Thus, S+ deliveries were more likely to be cut short for the observing response producing the relatively longer duration S+. The impact of this shortening of the longer S+ would be especially pronounced with the longest S+ durations. As a result, sensitivity was likely decreased because the obtained ratios of S+ durations were shorter than the arranged ratios. Unfortunately, Dinsmoor et al. did not provide obtained S+ durations. Despite this shortcoming of the data, the analysis in Figure 4 suggests that choice shows some sensitivity to relative conditioned-reinforcement magnitude.

Fig 4. A reanalysis of data from Dinsmoor et al. (1981).

Log observing-response ratios are presented as a function of log ratios of programmed S+ durations. Regressions of Equation 2 with S+ duration substituted for S+ rate are shown with slope (a), intercept (b), and variance accounted for (r²). The reanalysis was based on tabled data provided in the original publication. Filled and empty data points represent data from Squads 1 and 2, respectively.

Given that the present procedure was similar to that used by Dinsmoor et al. (1981), it is appropriate to ask whether a similar disproportionate shortening of S+ deliveries produced by the richer observing response might have impacted sensitivity. To evaluate this possibility, we examined the obtained average duration of individual S+ deliveries provided by the left and the right observing responses. Across pigeons and conditions, the average durations of S+ deliveries produced by the left and right keys were 13.7 s and 13.8 s, respectively. Thus, across conditions S+ deliveries were slightly shorter than the programmed duration of 15 s. Regardless, across conditions the ratio of obtained average S+ durations for the rich and lean observing keys was always near 1.0. The greatest deviation from a ratio of 1.0 obtained for any pigeon in any condition was 0.07. Thus, differences in obtained relative duration of S+ probably did not impact the allocation of observing responses.

Finally, in addition to providing a method to examine conditioned reinforcement, observing responses have been considered analogous to attending to stimuli that are to be discriminated (Dinsmoor, 1985; Wyckoff, 1952). Thus, the concurrent observing-response procedure used in the present study might be interpreted as demonstrating the role of conditioned reinforcement rate in the allocation of divided attention. From this perspective, the present results would suggest that the relative allocation of attention matches the relative rate at which that attending produces contact with an S+. This conclusion is consistent with the finding in human-vigilance studies that the relative distribution of behavior to two locations (e.g., time allocation or eye movements) matches the relative rate of detecting a signal at those locations (Baum, 1975; Schroeder & Holland, 1969). One question for future research might be whether the matching law also can be extended to the relation between the allocation of attending to the elements of compound stimuli and the relative presentation rate or value of those elements.

Appendix

Raw data summed across the final 6 sessions of each condition. Shown are the subject number, condition, total responses to the food key during S+ and for an S+ produced by left and right observing responses separately, total mixed-schedule responses on the food key and responses separated for VI and extinction during the mixed schedule, observing responses to the left and right keys, S+ deliveries produced by the left and right observing keys, total foods during S+ and foods during an S+ produced by the left and right observing keys, total foods during the mixed schedule, total time spent in S+ and during an S+ produced by the left and right observing responses, and total time in the mixed schedule and time in VI and extinction during the mixed-schedule.

Subject	Cond	Responses								S+ presentations		Food				Time
		S+ food key			Mixed food key			Obs				S+			Mixed	S+			Mixed
		Total	From left	From right	Total	VI	EXT	Left	Right	From left	From right	Total	From left	From right		Total	From left	From right	Total	VI	EXT
225	1∶1	3810	1852	1958	7128	3782	3346	7623	8742	175	182	75	33	42	34	82.93	41.05	41.87	277.07	97.57	179.50
	1∶3	4469	1103	3366	14220	6963	7257	4300	9885	87	263	58	14	44	58	79.97	20.18	59.80	280.03	103.78	176.25
	1∶9	4733	543	4190	13396	6290	7106	3665	9350	40	307	67	10	57	45	79.38	9.27	70.11	280.62	98.37	182.25
	3∶1	6891	5109	1782	14869	7185	7684	9082	3156	259	93	67	46	21	55	80.98	59.78	21.20	279.02	103.77	175.25
	9∶1	7408	6642	766	7762	5298	2464	14302	1934	335	38	83	76	7	34	85.27	76.36	8.91	274.73	98.73	176.00
	1∶9	6471	920	5551	9771	5809	3962	4602	8644	47	310	77	9	68	40	81.07	11.16	69.91	278.93	104.43	174.50
270	1∶1	8460	3864	4596	23166	10715	12451	3401	3618	135	154	49	27	22	66	67.26	31.71	35.55	292.74	115.49	177.25
	3∶1	6139	4330	1809	18587	10871	7716	6107	2921	196	77	53	38	15	62	63.89	46.05	17.84	296.11	119.86	176.25
	9∶1	4205	3414	791	21809	12813	8996	4605	1734	190	40	43	38	5	71	51.78	42.89	8.89	308.22	127.72	180.50
	1∶3	4189	1296	2893	21965	14529	7436	1738	2630	78	161	53	11	42	71	55.33	17.64	37.69	304.67	139.17	165.50
	1∶9	5376	648	4728	20905	14222	6683	1942	4313	33	252	66	7	59	56	66.40	7.96	58.44	293.60	125.85	167.75
	9∶1	4274	3430	844	25728	16154	9574	3403	1591	166	43	35	28	7	78	48.54	38.97	9.57	311.46	135.71	175.75
622	1∶1	6391	2920	3471	12241	4890	7351	6624	7704	179	206	80	43	37	41	89.29	40.33	48.96	270.71	97.96	172.75
	1∶3	6738	1329	5409	11219	3814	7405	2652	11346	79	318	66	18	48	47	90.50	17.94	72.56	269.50	92.25	177.25
	1∶9	8336	636	7700	12687	4695	7992	1570	11278	31	368	73	7	66	44	92.41	7.29	85.12	267.59	96.34	171.25
	1∶1	8563	3907	4656	14720	5645	9075	3865	6808	179	192	67	35	32	47	84.14	39.76	44.38	275.86	93.86	182.00
	3∶1	7061	4984	2077	16118	6238	9880	5495	4139	252	98	60	47	13	50	79.97	58.05	21.92	280.03	97.03	183.00
	9∶1	5937	5172	765	16896	6465	10431	7822	1118	319	43	53	46	7	59	81.81	71.62	10.19	278.19	96.19	182.00
199	1∶1	7622	2566	5056	14092	7095	6997	1941	3497	104	204	62	19	43	63	70.97	23.14	47.82	289.03	124.03	165.00
	3∶1	6733	4923	1810	13367	6539	6828	4407	1911	214	78	56	45	11	57	66.28	48.81	17.47	293.72	110.47	183.25
	9∶1	5476	4748	728	12518	6528	5990	3314	1779	221	37	45	36	9	66	58.31	50.24	8.08	301.69	119.94	181.75
	1∶3	6232	2117	4115	12453	5749	6704	2419	4892	95	183	56	22	34	44	61.98	20.41	41.57	298.02	104.77	193.25
	1∶9	6690	1221	5469	15561	7425	8136	1634	5217	42	227	41	10	31	58	61.09	10.07	51.03	298.91	106.66	192.25
	1∶1	6441	3177	3264	14170	7085	7085	3009	3769	144	142	48	26	22	65	64.97	32.07	32.89	295.03	111.03	184.00