Response Strength in Extreme Multiple Schedules

Abstract

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.

The strength of responding maintained by reinforcement schedules is frequently studied in procedures where two or more schedules of reinforcement, normally variable-interval (VI) schedules, are presented during experimental sessions. Common measures include steady state response rate, achieved after many sessions of training, and resistance to change, studied by disrupting stable responding in several test sessions and measuring response rate during these sessions as proportions of baseline.

It is widely accepted that baseline performance in both multiple and concurrent schedules is well described by a power function relating ratios of response rates to ratios of reinforcer rates (Baum, 1974; Lander & Irwin, 1968; Staddon, 1968; see Davison & McCarthy, 1988, for review). In logarithmic form, this relation is linear:

The terms B and R in Equation 1 refer to response rates and reinforcer rates, respectively, and their subscripts refer to the discriminated alternatives in the procedure. The parameter a measures the sensitivity of log response ratios to changes in log reinforcer ratios. Log c reflects the fact that responding may be higher overall for one schedule for reasons that have nothing to do with relative reinforcement (e.g., stimulus preferences), and is termed response bias. For concurrent VI VI schedules with pigeons as subjects, where the two alternatives are available simultaneously, the sensitivity parameter normally lies in the range 0.8–1.0. Such values indicate slight “undermatching” of response ratios to reinforcer ratios (Baum, 1979; Davison & McCarthy, 1988). For multiple schedules, where different schedules are presented successively, strong undermatching is observed. A review by McSweeney, Farmer, Dougan and Whipple (1986) reported a median value for a of 0.46. Of those cases in simple multiple schedules where reported variance accounted for was at least 80%, 79.8% of a values fell in the range 0.20 to 0.72.

The range of reinforcer ratios studied in concurrent schedules is normally less than 2 log (base 10) units. Sutton, Grace, McLean and Baum (2008) surveyed studies of concurrent VI VI for the purpose of a quantitative review, and further analyses of their data set show that omitting two studies (discussed below), 92% of all 885 obtained log reinforcer ratios were within the range −1 to +1. Recently, however, there has been debate as to whether the power function is adequate after all for concurrent schedules (Baum, Schwendiman & Bell, 1999; Davison & Jones, 1995; Sutton et al., 2008), based on data from a wider range of reinforcer ratios. The answer to this question may illuminate discussion of models that are more theoretical than descriptive in that they specify behavioral mechanisms that produce the performances observed at steady state. For example, Davison and Jones arranged extreme reinforcer ratios for pigeons responding under concurrent schedules, reported deviations from Equation 1, and showed that the pattern of deviation supported a theory in which choice ratios depend upon the discrimination of response–reinforcer contingencies operating at the two alternatives (Davison & Jenkins, 1985). By contrast, Baum et al. (1999) suggested that residuals of Equation 1 in their similar experiment were more consistent with a “fix-and-sample” pattern of responding on the rich alternative and occasionally visiting the lean alternative, a process suggested by foraging theory (Houston & McNamara, 1981). Although there is disagreement about the exact nature of deviations from Equation 1, both studies claim support for alternative models based on analysis of the residuals of Equation 1 when it was applied to data from reinforcer-ratio conditions that varied over a much wider range than had been used in previous research.

Research has also suggested several models of multiple schedule performance, and further development may be guided by research of the same kind as Davison and Jones' (1995) and Baum et al.'s (1999) work. Indeed, existing models for multiple schedule performance all predict slight deviations from the linear function predicted by Equation 1. One example is Herrnstein's (1970) equation for absolute response rate for a given schedule component:

and

The parameter m quantifies the extent to which reinforcer rates in the other component interact with those in the target component in controlling response rate. R_e is a term for the efficacy of the reinforcers maintaining Responses B₁ and B₂, which is determined by the availability of other, extraneous reinforcers in the situation, or manipulations of reinforcer size or quality. The parameter k is the maximum rate of responding in either component, constrained by the topography of the response under study. Equation 2 describes performance in multiple schedule components very well, but several authors have noted theoretical problems and proposed alternatives. For example, Williams and Wixted (1986) proposed that R₁ and R₂ in the denominator of Equation 2 should be the weighted average of reinforcers in the two components, (R₁ + mR₂)/(1+m), and that m be expressed separately for components that precede and follow a target component. McLean and White (1983) proposed that m in Equation 2 be set at zero, and that evidence of interaction among components be explained in terms of changing relative values for R_e across components. Despite these revisions, the resulting models all have a very similar form to Equation 2.

Dividing Equations 2a and 2b gives an expression for relative response rate, but it is clearly not a power function:

Figure 1 shows predictions from Equation 3, presented as log ratios, for 12 log reinforcer ratios over a range of 4 log units (parameter values used for prediction are given in the figure). With different values for corresponding parameters, identical predictions result from each of the models by McLean and White (1983) and Williams and Wixted (1986). Also given in Figure 1 is the best-fitting plot of Equation 1. Log response ratios predicted by Equation 3 deviate systematically from Equation 1. As the right panel shows, the residuals do not suggest random error in that they may be described as a third-order polynomial with a positive cubic coefficient (see also Baum et al., 1999; Sutton et al., 2008). Previous work has not attempted to identify such deviations in multiple schedules, possibly because, as with concurrent schedules, the range of reinforcer ratios studied has been too narrow. We know of only one study that used reinforcer ratios outside the range −1 to +1 (Reynolds, 1963), and those data are insufficient to assess Equation 1 because scheduled reinforcer rates ranged asymmetrically from about −1.5 to +0.5.

Fig 1.

Logarithms of ratios of response rates predicted by Equation 3 for a wide range of log reinforcer ratios. The solid line is a least-squares regression line, the equation of which is given in the figure (left panel). Residuals of the fitted regression line in the left panel, plotted as a function of log reinforcer ratios (right panel).

The power function of Equation 1 has also been used with resistance to change instead of response rate as the measure of response strength in multiple schedules (Nevin, 1992b), although only as a first approximation. To measure resistance to change, responding is maintained in components of a multiple schedule to establish a stable baseline, and is then disrupted over several sessions with variables such as delivery of noncontingent reinforcers during experimental sessions, or food given in the home cage immediately prior to them. Response rate in these test sessions is expressed as a proportion of baseline for each component, and has frequently been found to be greater for the richer schedule of the pair. Resistance to change in one component relative to that in the other component has sometimes been expressed as the ratio of the reciprocals of slopes of functions relating log response rate during a resistance test to the value of the disrupting variable. As Nevin (1992b) showed, these ratios appear to be a power function of reinforcer ratios, and again, the value of the exponent was around 0.4. A review of more recent data, by Nevin (2002), suggested a value of 0.5, where relative resistance was measured as the difference in log proportions of baseline between multiple-schedule components as in the analyses presented below. As with response rate, however, the range of reinforcer ratios studied with relative resistance to change as the variable may have been too narrow to ascertain whether Equation 1 really provides an adequate description.

The goal of the present study was to test the adequacy of Equation 1 as a description of multiple-schedule performance over a wide range of reinforcer ratios, and compare predictions from theoretical models. Reinforcer ratios were varied over a 4-log-unit range, similar to Davison and Jones (1995) for concurrent schedules, to improve detection of nonlinearity. Response strength was assessed in each condition using both steady-state response rate and resistance to change. The relations of each measure to reinforcer ratios were assessed against predictions from Equation 1.

METHOD

Subjects

Four experimentally naïve homing pigeons were maintained at approximately 85% of their ad lib weights. They were housed individually in a vivarium with a 12h∶12h light/dark cycle (lights on at 0600). Water and grit were always available in home cages, where supplementary feed of mixed grain was given after daily experimental sessions if needed to maintain their prescribed weights.

Apparatus

Four 3-key experimental chambers were used. They were 32 cm deep × 34 cm wide × 34 cm high, and enclosed in a sound-attenuating box. Masking noise and ventilation was provided by fans connected by plastic trunking to the boxes. One internal wall was an interface panel on which the three keys were mounted 21 cm above the wire floor. The center key could be illuminated from behind by red and green light-emitting diodes (LEDs). A force of approximately 0.15 N was sufficient to operate it, and turned off the LEDs for 50 ms. The other two keys were not used. A houselight was located above the center key, close to the ceiling. During reinforcement, a grain hopper was raised to an aperture in the panel, 6 cm above the floor, and illuminated with white light. The key light was also extinguished during reinforcement, which lasted for 3 s. Experimental events were arranged and recorded by a microcomputer and MEDPC® interface in an adjoining room.

Procedure

Subjects were autoshaped to peck lighted keys, and when responding reliably, were transferred directly to their first experimental condition. Experimental conditions were a series of multiple schedules, in which the components were presented for 90 s at a time (not counting reinforcement time) and alternated randomly with the constraint that no more than three consecutive presentations of a single component were possible. Daily, 2-hr sessions included 78 presentations of each component and were conducted at approximately the same time each day. The reinforcer rates arranged for red and green components are given in Table 1, along with the orders in which each subject experienced them. The reinforcement schedules are described below.

Table 1.

Random-interval schedules in each component of each condition, and responses per minute (B) and reinforcers per hour (R) for each bird. At the bottom, condition numbers are listed in order of exposure for each bird, along with the number of sessions of training given in each.

Training continued in each condition for at least twenty 2-hr sessions and until performance was stable to visual inspection over five consecutive sessions. Extended training was given in some conditions because of equipment failures or experimenter errors with prefeeding tests or running the subjects with the wrong program. (Stability will be examined in some detail in the Results section.) When performance was stable for all subjects, resistance to change (RTC) tests were conducted by prefeeding subjects with mixed grain 1 hr before experimentation for five or six consecutive sessions. Thus, with few exceptions late in the experiment, all subjects were tested across the same days. The amounts of food given were, in order, 20 g, 30 g, 30 g, 40 g, 40 g for all conditions, and for most, the series ended with a single session with 50 g food. During test sessions, food cups in home cages were emptied of any food left uneaten. Subject G2 was slow to autoshape, had completed only 14 sessions when the remaining birds were stable in their first conditions and its performance became unstable around its 16^th session. This subject's first test was delayed until the second test was conducted for the remaining subjects. Late in the experiment, this subject was two conditions behind the remaining 3 birds. The time spent completing the last two conditions for G2 was used to repeat the first conditions in the series for Subjects G1, G3 and G4, using more sessions of training than had been used in their first exposures. These repeat conditions were conducted without prefeeding tests.

Reinforcement Schedules

Because extreme reinforcer ratios were programmed for some conditions, very low reinforcer rates were sometimes required for one of the components. With standard length experimental sessions (often about 1 hr), VI schedules are likely to arrange a highly variable number of reinforcers per session. The likely result is that there would sometimes be extended runs of sessions with few or zero reinforcers in a component, and occasional sessions with many more than the average. We were concerned that behavior might come to track recent changes in reinforcement, and never properly stabilize at a rate that reflects the overall rate experienced over many sessions. To reduce this problem, we used 2-hr, rather than the normal 1-hr sessions. We further reduced it by using custom random-interval (RI) schedules that constrained session-to-session variability in obtained numbers of reinforcers. The actual schedule values used in each condition are given in Table 1.

The schedules constrained the number of reinforcers arranged in individual sessions to a defined range, eliminating occasional sessions with unrepresentative numbers of reinforcers, and the total number arranged within blocks of six sessions was fixed. The operation of these schedules is described in detail in Appendix A. Because this arrangement is different in several minor respects from both VI scheduling and standard RI scheduling, its performance is compared in Appendix A with simulations of these more standard schedules. These comparisons show that aside from controlling the absolute and relative rates of reinforcement across blocks of six sessions, the arrangement of reinforcers is very similar to more standard scheduling.

RESULTS

Table 1 gives the rates of responses and reinforcers in each component and condition, for each subject. These rates were averaged over the last six sessions in each condition, and form the basis of all subsequent analyses. The response ratio results are given in Figure 2. The equations of least-squares regression lines are given for each subject, and their slopes give values for a in Equation 1. Relative to the range reported by McSweeney el al. (1986), all values of a are high and considerably higher than the reported median of 0.46. They are within the range typically found for concurrent, rather than multiple, schedules (Baum, 1979). However, inspection of Figure 2 reveals a pattern in the residuals of these fits, such that data points closer to the origin (i.e., less extreme reinforcer ratios) suggest reduced sensitivity to reinforcement. Additional regression analyses found that when fitted to the conditions with log reinforcer ratios in the range −1.05 to +1.05 (the range studied extensively in the past), the slopes of fitted lines were indeed lower. Figure 3 gives the slopes of regression lines fitted to results from several different ranges of log reinforcer ratios, determined by progressively reducing the number of data points, symmetrical around the origin, that contributed to the analysis. Results from regressions using all 12 (10 for G2) data points, shown in Figure 2, have a mean value of 0.88. A narrower range of reinforcer ratios (−1.8 to +1.8) encompasses 10 data points (8 for G2) and the results had a mean sensitivity value of 0.77. Data from the range −1.05 to +1.05 (8 or 6 data points) had a mean value of 0.58. For ranges that were narrower still, sensitivity values varied unsystematically. As Figure 3 shows, slopes increased for every subject as the range of reinforcer ratios contributing to the regression analysis increased. Also in every case, the error bars (standard errors of slopes) for the left-most and right-most data points did not overlap. Finally, we computed sensitivity values using results from only the four most extreme reinforcer ratio conditions, and found a mean value of 0.96 (1.06, 0.91, 0.99 and 0.89 for Subjects G1, G2, G3 and G4, respectively). All of these results support the impression from Figure 2 that the function relating log response ratios to log reinforcer ratios was nonlinear.

Fig 2.

Log response ratios plotted as a function of log reinforcer ratios for each subject. Solid lines are least-squares regression lines, the equations of which are given in each panel. Unfilled symbols are replications of early conditions taken after extended training for three subjects.

Fig 3.

Fitted sensitivity values (Equation 1) as a function of the range of log reinforcer ratios. Error bars are the standard errors of sensitivity estimates.

Because the number of training sessions to stability tended to increase as the experiment proceeded, and because all subjects' first two conditions were ones with relatively moderate reinforcer ratios, it is possible that the lower sensitivity values for data points in the middle of the plots in Figure 2 were partly the result of less training in the first two conditions, and that higher sensitivity would have resulted even with moderate reinforcer ratios if training had been extended. However, the replications that were done for Birds G1, G3 and G4, with more than 80 sessions of training in each case, do not bear this out. Replications are shown as unfilled symbols in Figure 2, and do not generally suggest greater sensitivity than the existing data points in the same area of the plot. Furthermore, repeating the regression analyses, omitting the first determinations in these two conditions, did not change the results. Slopes still increased for all 3 subjects in the manner shown in Figure 3 (0.70, 0.89 and 1.0 for G1, 0.5, 0.76 and 0.89 for G3, and 0.4, 0.67, and 0.8 for G4).

As a further check of nonlinearity, we conducted an analysis of residuals. For each subject and each condition, we subtracted predicted log response ratios from observed log response ratios. Predicted log ratios were generated using Equation 1 with the parameter values given in Figure 2. Residuals, pooled across subjects, are plotted against predictions in Figure 4 and strongly suggest a function like that in the right panel of Figure 1. To confirm this, a single hierarchical regression analysis was performed in which residuals were regressed on Equation-1 predictions (first) and cubes of predictions (second). The cubic component was added to this regression analysis because, if it is a statistically significant predictor of the residuals, it would confirm the sinusoidal pattern in the residuals (and hence, sinusoidal deviations from the model's predictions). The analysis revealed that the model was significant overall, F(2, 43)  =  20.69, p<.01, and that the addition of the cubic component produced a statistically significant increase in variance accounted for, R²_inc  =  .49, F(1,43)  =  41.38, p<.01. The coefficient for the cubic component, 0.145 (SE .023), was statistically significant, t  =  6.43, p < .01, indicating a significant deviation from the linear function given by Equation 1.

Fig 4.

Residuals of fits of Equation 1 to log response ratios, plotted as a function of predicted log response ratios. Individual subjects' data are identified by different symbols.

Stability?

The asymptotic data described above were selected for each condition, and tests of RTC were started, when (1) at least 20 sessions of training had been given in a condition and (2) all subjects' log response ratios appeared stable across sessions to visual inspection (i.e., no visible trends across 5 sessions, and little variability from session to session). Because our conclusions might depend partly on the selection of data to represent stable performance, it is important to examine subjects' performances across sessions so the appropriateness of this selection can be assessed. Figure 5 shows log response ratios in each condition, omitting data from RTC tests, for all subjects. Log response ratios were calculated for blocks of three consecutive sessions (so that individual points could remain visible), and are shown as circles. Squares identify the sessions over which data were taken as stable, the data from which are given in Table 1. Vertical, dashed lines in each panel give the approximate points at which equipment or operator faults disrupted training. Except when these occurred early in a condition, these disruptions normally resulted in a decision to extend training. In one case, because of a response-key problem for G4, baseline sessions were reduced from six to five for all subjects. As explained earlier, Subjects G1, G3 and G4 completed the planned 10 conditions and replications of 2, whereas G2 started later than the rest, completed only the planned 10 conditions, and was trained in the final condition until the experiment ended.

Fig 5.

Log response ratios calculated for consecutive blocks of three sessions for the entire experiment. White squares give the values taken as asymptotic performance in each condition. Vertical dashed lines show when equipment or experimenter errors occurred. These normally resulted in extended training.

Clearly, performances were sometimes stable earlier than data were taken, particularly in the conditions with moderate, as opposed to extreme, reinforcer ratios. In these cases, however, it is also clear that log response ratios did not change systematically with more extended training. For the more extreme conditions, because of the scale of Figure 5, there are some suggestions of upward or downward trends shortly before performance was judged stable. (Note that the apparent trends tend to suggest that log response ratios even more extreme than those presented in Figure 2 might have obtained with further training, and if true, this would provide even stronger evidence of nonlinearity of the matching functions.) Overall, Figure 5 shows that the data used were representative of stable performance, and the nonlinear trends in Figure 2 were not the result of fortuitous selection.

Resistance to Change

Figure 6 shows group average results for 6 of the 10 RTC tests conducted (individual data, from all tests, will be presented below in a quantitative analysis). Responses in test sessions for each component were expressed as proportions of baseline. Where two test sessions were preceded by the same prefeeding amount, response rates were averaged across those sessions. Results with tests using 50-g prefeeding were omitted, because there were only three conditions with these data available for all 4 subjects. Figure 6 shows the expected differences in resistance to prefeeding between components, in that proportion of baseline measures were generally greater for the relatively rich component, and differed more strongly when the log reinforcer ratio was more extreme (+/− 1.0) than when it was more moderate (+/− 0.42). However, these generalizations apply less clearly to conditions with the most extreme log reinforcer ratios, where the average proportion of baseline measures were similar or identical for the two components.

Fig 6.

Proportion of baseline response rate observed in prefeeding tests in six selected conditions, calculated using group average response rates and plotted as a function of amount of food given.

To explore resistance to change results further, we related RTC to reinforcement obtained in the two components. We averaged response rates in each component across all test sessions, expressed these averages as proportions of baseline, and then computed relative RTC. Data from test sessions using 50-g prefeeding were used in this case, because these results were not averaged across subjects. The computation used the formula adopted by Grace and Nevin (1997): relative resistance  =  log (B_x1/B₀₁) − log (B_x2/B₀₂); Nevin (2002) showed that this measure was additive with respect to different disruptors and met the criteria for ratio scale measurement. B_x in this expression is response rate in the RTC test, B₀ is that in baseline, and the subscripts 1 and 2 identify the two components of the multiple schedules. These response rates are given in Appendix B.

Figure 7 shows relative resistance to change plotted as a function of log reinforcer ratios for all conditions, separately for each subject. Relative RTC has been analysed as a linear function of log reinforcer ratios in previous research (e.g., Nevin & Grace, 2000; Grace, Bedell, & Nevin, 2002). The results shown in Figure 7 are rather variable, but confirm that relative resistance generally increases with log reinforcer ratio. As was the case with response rate, however, there are indications that the function may be nonlinear. To test this, we conducted a hierarchical multiple regression analysis in which relative resistance was regressed on log reinforcer ratios (first) and log reinforcer ratios cubed (second). This analysis, performed on all subjects' data at once, produced a significant model overall, F(2,37)  =  17.68, p<.01. and there were significant contributions from both log reinforcer ratio (B  =  .34, p<.01) and log reinforcer ratio cubed (B  =  −.06, p<.01). Adding the cubes of log reinforcer ratios to the model, which introduces a nonlinear component to the function relating relative resistance to log reinforcer ratios, improved variance accounted for by 10.4% (p < .01). Predictions from this model are shown by the solid line in the right panel of Figure 7, along with group average relative RTC.

Fig 7.

Relative resistance to change for each subject and condition, calculated using Grace & Nevin's (1997) formula and plotted as a function of log reinforcer ratio (left panel). Group average relative resistance to change plotted as a function of group average log reinforcer ratio (right panel). The solid and dotted lines in the right panel show predictions from multiple regression analyses described in the text.

The pattern evident in the right panel of Figure 7 was generally evident in the individual subjects' data in the left panel. The main exception is that only 3 of the 4 subjects showed a decrease in relative RTC from the first to the second data point in the graph (G2 showed an increase). All 4 subjects showed a decrease between the second-to-last and last data points in the figure. Thus, the significant cubic component in the group analysis suggests that the function relating relative RTC to relative reinforcer rate may indeed be nonlinear when extreme conditions are included.

DISCUSSION

The main result was that log response ratios were a nonlinear function of log reinforcer ratios, contrary to expectations from the generalized matching law. This was supported by analyses showing that Equation 1 sensitivity values decreased as the range of contributing log reinforcer ratios was narrowed around 0. It was also supported by multiple regression analyses, conducted on the residuals of Equation 1, that confirmed the residuals were not random and revealed a systematic pattern. The experiment was typical of multiple schedule research in most respects, the exceptions being the wider range of reinforcer ratio conditions studied, and the manner in which reinforcers were arranged.

The fact that our performances over the normally-studied range of reinforcer ratios (mean sensitivity 0.58) were reasonably close to the median reported by McSweeney et al. (1986) suggests that, at least in nonextreme conditions, our method of scheduling reinforcers produced performances similar to those found with standard VI (or RI) scheduling. However, the same may not be true for extreme conditions, where one of the two components arranged very low rates of reinforcement. Unless very long sessions are used, standard schedules are likely to arrange highly variable numbers of reinforcers from session to session in the leaner component. Our method was adopted to maintain fairly tight control over obtained reinforcer ratios, by controlling session-to-session variation in numbers of reinforcers obtained and thereby restricting variation in both absolute and relative reinforcer rates. Control of reinforcement was important because in the most extreme conditions, quite small differences between the intended and obtained rates can make large differences in the reinforcer ratio. The resulting variability might have obscured the systematic deviations from linearity we were testing for.

One way that variability might obscure the function is by introducing noise in measurement of reinforcer ratios. Even the response ratios we observed might not, if plotted against log reinforcer ratios that were poorly measured at extreme values, appear systematically nonlinear. A second way variability in reinforcement might obscure trends in log response ratios is by compromising measurement of absolute response rates, particularly in the leaner component of extreme conditions. With our method, the arranged rate of reinforcers is variable within a six-session block, but the average for the block is equal to the overall rate. When reinforcers are arranged by VI schedules, the rate measured over six sessions can vary considerably across successive blocks (see Appendix A), and it seems likely that response rate will tend to track recent changes in reinforcer rate. The assumption that response rates measured at steady state are a reflection of the overall reinforcer rate becomes questionable if reinforcer rates are very low; responding may virtually cease after several sessions with no or few reinforcers, be “reinstated” and then increase sharply in a session with many more reinforcers than the average, never properly stabilizing at a rate that reflects the overall reinforcer rate. For these reasons we suggest that controlling the arranged rate of reinforcers, at least across blocks of several standard-length sessions, may be important in determining response rates accurately when reinforcement is rare.

The nonlinear functions relating log response ratios to log reinforcer ratios in Figure 2 are consistent with the expectations from models of multiple schedule performance, shown in Figure 1. However, these models were not found to fit the data well. Figure 8 shows Equation 3 fitted to log response ratios in the present experiment. Baseline response rates were first averaged across replications for Subjects G1, G3 and G4, and then across subjects. Then, Equation 3 was fitted to log ratios of group average response rates using Microsoft Solver ®, (m  =  0.59, R_e  =  0; VAC 0.97; see below for fitting details) and predicted log response ratios are shown along with the data. Equation 3 clearly does not capture the curvature in the observed log response ratios. Fitting an equation similar to Equation 3 but derived from Williams and Wixted's (1986) model yields identical predictions. Although these models can predict the general nonlinear form that was observed, this prediction occurs only when there is undermatching. When m approaches 1.0, implying strong interaction between components and hence matching, log ratios of predicted responses rates from both models become linear functions of log reinforcer ratios. That is, with a large m value predictions miss the comparatively shallow part of the function close to the origin, and with a smaller m they miss the matching observed at extreme reinforcer ratios. The same thing occurs if R_e in Equation 3 (or Williams & Wixted's C) is varied; increasing it makes the function steeper but straighter, and decreasing it makes it curvilinear but shallower, missing the results from extreme conditions. Thus, neither can provide a good account of the present data, where there is a nonlinear function that reaches matching only at extreme reinforcer ratios.

Fig 8.

Fit of Equation 3, and a similar equation derived from Williams and Wixted's (1986) model, to log ratios of group average response rates.

The failure of both models is probably related to a problem that became apparent soon after the publication of Herrnstein's (1970) quantitative treatment of the Law of Effect, namely, the manner in which interaction of components was represented mathematically. Values for m (Equation 3) that are less than 1.0 account for both the weaker component interactions found in multiple (as opposed to concurrent) schedules, indicated by the smaller size of behavioral contrast effects, and weaker discrimination in multiple schedules, indicated by undermatching. Both models account for these common quantitative features of multiple-schedule performance using relatively small values for m. Such values for m raise other problems for Equations 2–3, however, as was quickly noticed by Spealman and Gollub (1974; see McLean & White, 1983, for a review of problems that arise when m < 1). Some are addressed by Williams and Wixted's (1986) model, but the present data pose problems for that formulation also. McLean and White suggested a version of Equation 3 in which R_e was the only free parameter (m is set at zero). Their model does not suffice either, however, as increasing R_e results in a function that is closer to linear, as in Equation 3. They further proposed that component interaction was mediated by reallocation of extraneous reinforcers (McLean, 1992; 1995), and some form of reallocation function may serve to explain the present results. No such function has yet been developed, however.

As far as we know, the only published model for multiple schedule performance that adequately describes the present results is one by McLean (1991). In this equation, as in Herrnstein's theory, the animal distributes the stream of behavior that occurs during a component between two classes of activity; responding and other (“extraneous”) behavior, and this allocation depends on the relative reinforcement obtained for the two classes in that component. Denoting responding and other behavior B and B_e, respectively, and reinforcers for these activities as R and R_e respectively, McLean's (1991) model (for Component 1) is:

A similar equation is written for Component 2. The constant a is sensitivity to the ratio of reinforcers concurrently available for responding versus other activities in Component 1, c is bias, and n is an interaction term. Where McLean's (1991) model differs from previous formulations is the mechanism for interaction. With n < 0, the allocation to responding in Component 1, B₁/B_e1, decreases with increases in the corresponding reinforcer ratio in the other component, R₂/R_e2. Thus, contrast effects are predicted in behavior allocations. In an experiment with explicit reinforcers for an analog of “extraneous” behavior (pecks to a second response key), McLean (1991) used responses to and reinforcers from the second key for values for B_e and R_e, and found that during the first 25 s of 100-s components, fitted values for n were indeed negative. As components progressed, however, n increased, and the fitted values for whole-of-component responding were, across subjects, approximately zero. Although McLean (1991) concluded that Equation 4 captured only local contrast effects in his data, we assume here that the mechanism it specifies for component interaction is more general.

Although Equation 4 predicts within-component ratios of responding to other behavior, it can be used to predict multiple-schedule response ratios (B₁/B₂) with an additional assumption. Similar to Herrnstein (1970; 1974), McLean and White (1983) and Baum (2002), it is assumed that the stream of behavior that is distributed to produce an “allocation” results in a constant overall quantity of behavior per unit of time. That is, during the two components of a multiple schedule, B₁ + B_e1  =  B₂ + B_e2  =  k (a constant). Response ratios predicted for each component by Equation 4 (B₁/B_e1 and B₂/B_e2) can be converted to proportions (i.e., B₁/(B₁+B_e1) and B₂/(B₂+B_e2)) by dividing each ratio by itself plus one. Following the assumption above, the resulting proportions for Components 1 and 2 each have the same denominator (k) which cancels when the Component 1 proportion is divided by the Component 2 proportion, giving B₁/B₂. This method enables Equation 4 to be applied to the present data, using Microsoft Solver ®, as follows. Predictions from Equation 4 (and a similar equation for Component 2) were generated using starting values for n, c, and R_e, giving B₁/B_e1 and B₂/B_e2 for each condition (a was fixed at 1.0 and R_e1was constrained to equal R_e2). These ratios were converted to proportions as described above, and then divided to give predicted B₁/B₂. The logarithms of these predicted response ratios can then be assessed against the observed log response ratios, and Solver can be used to find the parameter values that minimize error in predictions. Figure 9 gives the results. Although there is some evidence of systematic deviations of data from the predictions, the fits are very good overall.

Fig 9.

Fits of Equation 4 to log response ratios for each subject. Values of free parameters in Equation 4 are given in each panel.

We conducted further analyses to compare this performance with that of Herrnstein's (1970) equation and of Equation 1, computing values for the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). A bias term (log c) was present in all models. In the case of Herrnstein's (1970) equation, whether using Equation 2 or 3, unique solutions cannot be found for both m and R_e unless the experimental design holds one component of the multiple schedule constant (not so here). That is, identical predictions are found with either parameter fixed and the other fitted. Accordingly, we fixed R_e at 0 reinforcers per hour (as did Herrnstein, 1970, see Fig. 13) and found values for m and log c that produced the best match of predicted log response ratios to observed log response ratios, using Microsoft Solver ®. The results are given in Table 2: fitted values for m, log c, and variance accounted for in log response ratios. The fits were very good, accounting for more than 94% of the variance in all cases. These fits surpassed those for Equation 1, primarily because predicted log response ratios with the fitted parameter values were a slightly nonlinear function of log reinforcer ratios. However, scatterplots (not shown) frequently revealed error in predictions at extreme log reinforcer ratios, similar to Figure 8. Recall that fits of Equation 4 were obtained with, a, concurrent sensitivity, fixed at its normative value of 1.0 and R_e1 constrained to equal R_e2. (These assumptions, perfect matching within components and equal R_e across components, are both implicit in Herrnstein's 1970 equation.) The parameters n, log c, and R_e were estimated by fitting to log response ratios. The resulting fits were very good, slightly better overall than those for Herrnstein's (1970) equation. At least 97.7% of variance in log response ratios was explained by Equation 4.

Table 2.

Results of model comparisons. Parameter values, variance accounted for in log response ratios, and Akaike and Bayesian Information Criterion values. Note that Re was fixed at zero for Equation 3, and a was fixed at 1.0 for Equation 4.

To address the difference in number of free parameters (Equation 4 has one more than either Herrnstein's 1970 equation or Equation 1), we computed the AIC and the BIC indices for all fits, using the equations given by Navakatikyan (2007). For the AIC, we used the AICc form recommended for small samples. AICc is a measure of model performance that decreases with increasing VAC and increases with increasing free parameters, and is often used to compare models when the number of free parameters differs. The BIC is similar, but more conservative. In each, the better model is the one with the lower criterion value. Comparing the AICc and BIC values across fits in Table 2, it is evident that Equation 4 consistently outperforms Herrnstein's (1970) equation and the Generalized Matching Law for 3 of the 4 subjects. The exception, Subject G2, is the individual whose data (Figure 2) shows least deviation from linearity, and for whom the fitted value for n is closest to zero (suggesting weak component interaction). One implication of this is that conditions that reduce the interaction of components (e.g., longer component durations, or timeout periods between components) may reduce or eliminate the nonlinear pattern observed in the present results. Overall, both models provide impressive accounts of the present results, but Equation 4 has an advantage in that the interaction mechanism it embodies captures the effects of component interaction on log response ratios slightly better than Herrnstein's m, at least when extreme reinforcer ratio conditions are used.

Whereas log response ratios were a reverse-S shaped function of log reinforcer ratios, relative resistance to change data from the present experiment suggest an S-shaped function, at least in the group-aggregate data. Thus, Equation 1 appears to be inaccurate in its application to both response rate and RTC when a wide range of reinforcer ratios is used. It is possible that these two deviations from expected linearity are related. Most RTC research has used relatively moderate reinforcer ratios, within the range that produced undermatching in the present study. Undermatching means that reinforcer probability (R₁/B₁ and R₂/B₂) is always greater in the richer component. When matching occurs, as it did with more extreme reinforcer ratios, these two probabilities are equal. If RTC is partly determined by reinforcer probability, then it would differ between components (favoring the richer one) most strongly in those conditions that produced undermatching, but would tend to converge in the more extreme conditions, where matching was observed. This would produce the nonlinear function for relative RTC that is apparent in Figure 7. As a test, we repeated the multiple regression analysis on group aggregate relative RTC results reported above, using obtained reinforcer-probability ratios in place of the cube of reinforcer ratios. The analysis again produced a statistically significant model, F(2,37)  =  15.04, p<.01, and the contributions of both log reinforcer rate ratios and log reinforcer probability ratios remained significant (B  =  .12, p < .01, and B  =  .28, p < .05, respectively). Again, adding the second predictor variable (log reinforcer probability ratio) significantly improved variance accounted for (6.4%, p < .05). Predictions from this model are shown in the right panel of Figure 7 as a dotted line. Because these predictions are partly dependent on observed baseline reinforcer probability values, the function is slightly less symmetrical than the one obtained using cubes of reinforcer ratios (solid line).

The idea that RTC might be partly determined by the ratio of reinforcers to responses in baseline reinforcer probability is consistent with the finding that it varies directly with reinforcer rates, and inversely with response rates, in the preceding baseline conditions. For example, Nevin, Grace, Holland and McLean (2001) examined RTC in VI and VR schedules. They controlled for baseline reinforcement rates in two experiments, the simpler method being yoking of the VR to the VI schedule. Their result, in both experiments, was that baseline response rate was higher in the VR component (meaning that reinforcer probability was greater in the VI component than the VR component). Resistance to change was higher in the VI component. Similarly, Nevin (1974) found that RTC tended to be higher when DRL contingencies were superimposed on VI schedules than when DRH contingencies were superimposed, generating higher baseline response rates, and hence lower baseline reinforcer probabilities, than in the DRL component (see also Blackman, 1968; Lattal, 1989; but also Fath, Fields, Malott & Grossett, 1983). Several other results including the effects of alternative reinforcement (e.g., Nevin, Tota, Torquato & Shull, 1990; Rau, Pickering & McLean, 1996), and reinforcement history (Doughty, Cirino, Mayfield, da Silva, Okouchi & Lattal, 2005) on RTC can be interpreted similarly in that each of these manipulations affect reinforcer probability, at least over the recent past.

It should be noted, however, that if resistance to change is indeed related to baseline reinforcer probability, the relation is not entirely straightforward. Nevin (1992a) and Grace, McLean and Nevin (2003) showed that RTC in a constant component is inversely affected by reinforcement in the alternated component of multiple schedules. Since response rate is similarly affected (behavioral contrast), reinforcer probability and RTC vary in opposite directions in the constant component. Explanation of contrast effects in both response rate and RTC thus requires reference to relative reinforcement—relative reinforcer rate for responding, and relative reinforcer probability for RTC. A greater problem arises when reinforcers are delivered at the same rate, but are delayed in one component of a multiple schedule. Grace, Schwendiman and Nevin (1998) and Bell (1999) showed that baseline response rate and RTC were both lower in a component with unsignalled delays to reinforcement than in another component where reinforcers were immediate. Reduced baseline response rate implies higher reinforcer probability, but RTC was reduced, not increased.

Equation 4 successfully predicted performances in baseline, and if it could also predict those in test sessions, perhaps with larger values for the R_e parameter to reflect devaluation of reinforcers by prefeeding, then it might provide an accurate characterisation of both response rate and RTC. However, Figure 10 below suggests otherwise. First, Equation 4 was fitted to group average data and predicted absolute response rates were taken (baseline). Then R_e was increased, leaving other parameters constant. The new predicted absolute response rates were then expressed as proportions of baseline, and relative RTC was calculated. The best fitting value for R_e was approximately twice that in baseline. These predictions, and obtained group average relative RTC, are both plotted against log reinforcer ratios. The correspondence is reasonably good, but it is clear that Equation 4 misses the downward trends in the actual data (and in the predictions shown in Figure 7) over the first and last pairs of data points in the plot.

Fig 10.

Group average relative resistance to change in each condition, plotted as a function of log ratios of group average reinforcer ratios. The solid line shows predicted relative resistance to change derived from Equation 4.

Equation 4, and other steady-state models for absolute or relative response rate, may fail to capture changes in resistance to change because they contain only terms for the conditions in effect at the time. They contain none for the immediately-preceding (baseline) conditions, and RTC may be sensitive to these. A particular problem may arise if baseline reinforcer probability affects RTC, as suggested above. When response rate is disrupted in, say, the first session of a RTC test, the ratio of reinforcers to responses changes, thereby changing the conditions that partly determine the magnitude of further changes in response rate. It may be that static models of the kind employed to describe steady state responding will not suffice for resistance to change and that a more dynamic approach is needed.

To conclude, the present experiment provides response rate and resistance to change results from multiple schedules with a much wider range of reinforcer ratio conditions than has been studied previously. The main result was that in steady-state performance, the strong undermatching characteristic of multiple-schedule response ratios was found, but was restricted to a range of reinforcer ratios spanning two log units. With reinforcer ratios outside that range (more extreme), matching was observed. The generalized matching law and three other models of multiple-schedule performance were assessed, and all provided excellent accounts of the results in that they accounted for more than 94% of the variance in log response ratios. They can be distinguished, however, on their ability to capture the nonlinearity in the observed function relating log response ratios to log reinforcer ratios. This nonlinearity is perhaps most problematic for the generalized matching law, where it implies inconstancy of the sensitivity parameter. Equations by Herrnstein (1970) and Williams and Wixted (1986) are capable of assuming this nonlinear shape, but cannot do so with near-matching in extreme conditions because the nonlinearity in their predictions relies on low component interaction, requiring strong undermatching even in extreme conditions. The model by McLean (1991) provided the best fits, albeit at the cost of an additional free parameter. AIC values suggest that this additional parameter explained sufficient additional variance in the data that this model has an advantage, possibly because of the way it handles component interaction.

Our second result was that relative RTC bore a nonlinear relationship to log reinforcer ratios, suggesting that Equation 1 provides only a broad characterization of RTC. Nevin (1992a) and Nevin et al. (1990) have pointed out that response rate models, including Herrnstein's (1970) equation, also fail to capture all of the results found in the RTC literature. Since these models perform rather well with baseline data, the shortcoming seems to lie in their ability to capture the immediate effects of disruptive variables such as prefeeding. The equation suggested by McLean (1991) proved most adequate to describe baseline response ratios across the wide range of conditions of the present experiment, and captured much, but not all, of the pattern observed in relative RTC.

Thus, existing models for multiple-schedule response rates have proven adequate to capture an impressive 98% of the variance in response ratios observed across stable reinforcement conditions. Determining how to extend them to deal equally well with the short-term effects on response rate of changes in the reinforcement environment, such as those in resistance to change research, remains a significant challenge but will bring the benefit of a comprehensive account of response strength.

APPENDIX A

Reinforcement Scheduling

Our reinforcement schedules operated as follows. A given number of reinforcers was assigned to each component at the start of each session. The number was determined each day for each component by sampling randomly, without replacement, from a list of six. The list was refreshed after a six-session cycle was completed. The position of each reinforcer within the session was then determined by taking one random number (between 0 and 1) for each reinforcer. These numbers gave the proportion of the time to be spent in the relevant component (3500s) that needed to elapse before the reinforcer became available. Once these were determined, proportions were converted to interreinforcer intervals and the schedule operated as a VI schedule would operate.

To illustrate, if three reinforcers were arranged for the red component, and the three random numbers were, say, 0.56, 0.33 and 0.61, then the first reinforcer of the session would become available after 1155s (.33*3500), the second 805 s after the first was delivered (0.56*3500-1155) and the third 175 s after that (.61*3500-(1155+805)). Note that no reinforcers were arranged for the last 10 s of the session, because of the risk that they might not be obtained. Except for the last component of a session, reinforcers arranged but not obtained at the end of a component were held until the next presentation of that component.

In this way, day-to-day variability in obtained reinforcement rate was constrained within the limits imposed by the list contents, and extended runs of sessions with zero reinforcers, or with much higher-than average rates, were avoided in the leaner components of extreme conditions. Ten such RI schedules were used in the experiment. Their overall values, and the associated lists containing reinforcer numbers for six sessions, were as follows: RI 3500 s (0,0,1,1,2,2), RI 2333 s (0,1,1,2,2,3), RI 437.5 s (6,7,8,8,9,10), RI 212.1 s (14,15,16,17,18,19), RI 140 s (23,24,25,25,26,27), RI 53.8 s (63,64,65,65,66,67), RI 47.6 s (72,73,73,74,74,75), RI 42.7 s (80,81,82,82,83,84), RI 39.5 s (86,87,88,89,90,91), and RI 39.3 s (86,87,88,90,91,92). To explain, when RI 3500 s was arranged in a component, successive blocks of six consecutive sessions (starting at the first session) would each contain two sessions with no reinforcers arranged, two with exactly one, and two sessions with exactly two reinforcers arranged—a total of six across the six sessions.

To enable comparison of our method and regular VI scheduling, we assessed the numbers of reinforcers likely to be obtained in sessions if VI schedules had been used, in Condition 1 (reinforcer ratio  =  1∶89). We simulated 10,000 “sessions” in which interreinforcer intervals were sampled randomly without replacement from 12-interval constant-probability VI schedules until the sum of their values exceeded the time for which the component was presented in our experiment. We then counted the number of intervals completed in the “session” (assuming, in effect, that reinforcers would be obtained immediately they became available). VI schedules were refreshed for each session.

We then calculated a moving average (of 6) of the number of reinforcers arranged for the RI 3500 s in 102 sessions for Subject G3 (i.e., our method). Figure A1 gives these results, along with the moving averages from a randomly-selected section of the same length from the output of the simulation. Each point shows the average number of reinforcers arranged in the last six sessions. Both records show variability, and occasional sustained excursions away from the overall rate (1.0). In the record for Subject G3, these are somewhat less extreme, and are noticeably briefer (the average returns to 1.0 at least every six sessions).

Fig A1.

Moving average of six sessions' reinforcers arranged per session, from a simulation of VI 3500 s. Also shown is the moving average actually experienced by 1 subject in the present experiment.

Figure A2 gives frequency distributions of log reinforcer ratios, produced by taking log ratios of the moving averages of the output from the entire simulation for VI 3500 s and VI 39.3 s. For comparison, the distribution is also shown for a 10000-session simulation using our method. With VI, even though based on six-session average reinforcers per session, log reinforcer ratios were undefined in a small percentage of cases. Our schedules less often arrange log reinforcer ratios less than −2.3 or greater than −1.8. Values close to the overall arranged value (−1.95) occurred more often with our method, because it returned the average in the leaner component to 1.0 every six sessions. Both of these analyses suggest that the likely effects of using our method were the intended increases in control over relative reinforcement.

Fig A2.

Relative frequencies of log reinforcer ratios calculated from reinforcer rates averaged across six sessions, from the output of a 10000-session simulation using VI 3500-s and VI 39.3-s schedules (unfilled bars). The filled bars show the results that would be obtained across 10000 sessions with our method of scheduling.

Because of the constraint on numbers of reinforcers that could be arranged in a session, our method of scheduling also differs from standard random-interval schedules, where a probability gate is interrogated periodically and reinforcement is scheduled according to a constant probability. Figure A3 shows that the interreinforcer intervals arranged using our method were comparable with those expected from a regular RI schedule. Relative frequencies of interreinforcer intervals (in 200-s bins) are plotted for each subject in the present experiment, for the leaner component in Conditions 1 and 2. Although there is considerable variability across subjects and bins, it is clear that the intervals arranged by our method are not unrepresentative of RI scheduling.

Fig A3.

Relative frequencies of interreinforcer intervals arranged in the leaner components of Conditions 1 and 2, in 200-s bins, for each subject. The solid line gives relative frequencies of the same intervals arranged by a regular RI schedule, in which a probability gate was interrogated every 0.5s.

To summarize, the method of reinforcement scheduling used here strongly resembles regular RI and VI scheduling. The most salient differences are apparent when absolute and relative reinforcer rates are examined across a small number (6) sessions, where both remain closer to the intended (overall) rates with our method, whereas greater variability is expected with VI or RI scheduling.

APPENDIX B

Responses per minute in baseline (B₀) from Table 1, and average responses per minute across all sessions of resistance to change tests (B_x), for Components 1 and 2 in each condition.