Stimuli, reinforcers, and the persistence of behavior

Abstract

This article reviews evidence from basic and translational research with pigeons and humans suggesting that the persistence of operant behavior depends on the contingency between stimuli and reinforcers, and considers some implications for clinical interventions.

The metaphor of behavioral momentum proposes that the rate of responding under constant conditions of reinforcement is analogous to the velocity of a moving object, and the persistence of that rate in the face of a challenge depends on the behavioral analog of physical mass. Thus, steady-state response rate and its resistance to change are independent aspects of behavior, just as velocity and mass are independent aspects of a moving object. Moreover, behavioral momentum theory proposes that response rate depends on operant response-reinforcer contingencies whereas resistance to change depends on Pavlovian stimulus-reinforcer contingencies (for review see Nevin & Grace, 2000).

Characterizing contingencies

Every serious practitioner of behavior analysis is familiar with the effects of contingencies between operant responses and reinforcers. Under the most basic operant contingency, a reinforcer is presented every time a designated target response occurs and is never presented in the absence of the response. This contingent relation can be weakened or changed in various ways. For example, the reinforcer may not be presented after every instance of the target response but only if some other condition is met, such as “if 10 responses have occurred” (Fixed ratio or FR 10), or “if 1 min has elapsed since the preceding reinforcer” (Fixed interval or FI 1 min). Another way to alter the basic contingency is to present reinforcers contingent on the nonoccurrence of the response; for example, if the target response does not occur for 10 s (Differential reinforcement of other behavior or DRO 10 s). Or the contingency may be abolished altogether by arranging that reinforcers are presented independently of the target response (noncontingent reinforcement or NCR); for example, at variable times averaging 1 min (VT 1 min). The effects of these contingencies – given that an effective reinforcer has been identified – are so well known and so repeatable that they can be illustrated under poorly controlled conditions in student laboratory courses.

The strength of the operant contingency may be varied by changing the probabilities of reinforcement given that a response occurs, or does not occur, in brief segments of time (Hammond, 1980). The contingency may then be expressed as the difference between those probabilities,)P (e.g., Wasserman, Elek, Chatlosh, & Baker, 1993). In the usual free-operant situation where time is not readily sliced into segments, the strength of the operant contingency may be captured by the proportion of all reinforcers that are contingent on the designated response.

Contingencies between stimuli and reinforcers can be specified and modified in similar ways. In the strongest contingency, a reinforcer is presented every time a brief stimulus is presented and not otherwise, a procedure that is familiar to students of introductory psychology as Pavlovian or respondent conditioning. The Pavlovian contingency can be modified in the same general way as operant contingencies: For example, the reinforcer may be presented after only some portion of the tones (partial reinforcement, analogous to a variable-ratio (VR) schedule for an operant), or it may be presented only in the absence of the tone (inhibitory conditioning, analogous to DRO for an operant response). In discrete-trial procedures, the stimulus-reinforcer contingency may be quantified by the probabilities of reinforcer presentation given that a designated stimulus is present, or is not present in an equivalent time sample, and then calculating the correlation coefficient phi (Gibbon, Thompson, & Berryman, 1974). An alternative proposed by Gibbon (1981) that is better suited to free-operant discrimination procedures is the ratio of the reinforcer rate in the presence of a designated stimulus to the overall reinforcer rate in the experimental setting.

Stimulus-reinforcer (hereafter Pavlovian) contingencies are of obvious interest to researchers studying autoshaped key pecking, conditioned suppression, or other forms of respondent conditioning, but may be neglected by practitioners of behavior analysis because their effects are often overshadowed by the sheer power of response-reinforcer contingencies to control behavior. However, Pavlovian contingencies deserve consideration because they are embedded in any application of operant contingencies that involves stimulus control. Indeed, they may operate to undercut some of the effects of operant contingencies that are widely used and highly successful in reinforcement-based interventions on clinical settings.

Basic research: Effects of added response-independent reinforcers

Here's an example from the pigeon lab involving a multiple schedule of reinforcement with different operant and Pavlovian contingencies in its components (Nevin, Tota, Torquato, & Shull, Experiment 1). As shown in Figure 1, there are two components, signaled by different colored lights on the pecking key, that last for fixed durations and alternate successively in time. In Component 1 (green), responses are reinforced after variable intervals averaging 1 min (VI 1 min) yielding 60 reinforcers/hr. In Component 2 (red), responses are reinforced according to the same VI 1-min schedule (60 reinforcers/hr) and, in addition, reinforcers are presented independently of responding at variable times averaging 30 s (VT 30-s), yielding an additional 120 reinforcers/hr. The operant contingency is stronger in the green, VI-only component because all reinforcers are response-contingent, whereas in the red, VI+VT component, only 1/3 of the reinforcers are response-contingent. Conversely, the Pavlovian contingency is stronger for the VI+VT component because the ratio of the reinforcer rate in that component to the total reinforcer rate in the experiment, (60+120)/(60+120+60), is three times the value of the ratio in the VI-only component [i.e., 60/(60+120+60)].

Figure 1.

A time-line diagram of a multiple schedule where reinforcers are available in both components according to VI 1-min schedules (60 reinforcers/hr) indicated by upper-case bold X's, and in one component, reinforcers are also presented independently of responding on a VT 30-s schedule (120 reinforcers/hr) indicated by lower-case x's.

To evaluate the effects of these contingencies, Nevin et al. (1990, Experiment 1) exposed pigeons to the multiple-schedule procedure diagrammed in Figure 1 for 30 sessions. All pigeons exhibited higher response rates in the VI-only than in the VI+VT component – a result that makes sense in relation to the difference in operant contingencies. The same result has been obtained with single VI schedules when response-independent reinforcers were introduced (Rachlin & Baum, 1972) and is well established in the basic research literature.

Nevin et al. (1990, Experiment 1) also arranged two tests of resistance to change: prefeeding, where the pigeons received 40-60 g of food 1 hr before selected daily sessions with the usual contingencies operating in both components, and extinction, where all reinforcers were discontinued in both components. The results are presented in Figure 2, averaged across pigeons because all exhibited the same effects. The panel on the left presents baseline response rates averaged over five sessions and the average rates of responding in five prefeeding sessions. As noted above, baseline response rates were higher in the VI-only component, but during prefeeding, response rates were higher in the VI+VT component. The same was true for extinction: As response rates decreased over successive sessions, extinction responding decreased later and to a lesser extent in the V+VT than in the VI-only component. Although the differences between components were not large, the reversal of ordering from baseline to prefeeding or to extinction was highly reliable. The conclusion is that resistance to change depended on the Pavlovian contingency, which was stronger in the VI+VT component, whereas baseline response rates depended on the operant contingency as noted above.

Figure 2.

Rates of responding by pigeons during baseline (BL) and prefeeding test sessions (left panel) and during baseline and extinction test sessions (right panel) in the multiple VI, VI+VT schedules diagrammed in Figure 1 (from Nevin et al., 1990, Exp. 1). Note that the order of response rates for components with and without added VT reinforcers reversed during the tests.

Since 1990, the finding of increased resistance to change in the presence of a stimulus correlated with added noncontingent reinforcers has been replicated with rats (Harper, 1999) including studies with qualitatively different VI and VT reinforcers (Grimes & Shull, 2001; Shahan & Burke, 2004); with goldfish (Igaki & Sakagami, 2004); and with college students (Cohen, 1996), so it has substantial generality.

Basic research: Effects of added reinforcers for alternative behavior

At a conference in 1988, Rick Shull told me that he and his students were getting similar results with pigeons in multiple concurrent VI schedules when the added reinforcers were contingent on an explicit alternative response. In Component A, a target response (pecking the right key) was reinforced on a VI 240-s schedule (15 reinforcers/hr) and an alternative response (pecking the left key) was reinforced concurrently on a VI 80-s schedule (45 reinforcers/hr). In Component B, the target response obtained 15 reinforcers/hr while the alternative response was not reinforced, and in Component C, the target response obtained 60 reinforcers/hr while the alternative was not reinforced. Thus, the ratios of reinforcers in Components A and C to overall reinforcer rates were the same in Components A and C [i.e., (15+45)/(15+45+15+60) and 60/(15+45+15+60)], and were four times greater than in Component B [i.e., 15/(15+45+15+60)].

As shown in Figure 3, baseline response rates were as predicted by the literature on concurrent and single VI schedules: The target response rate was about 1/3 of the alternative response rate in Component A, roughly matching the obtained ratio of reinforcers, and was substantially higher in Component B where no alternative reinforcers were available. Target response rate was highest in Component C, where its reinforcer rate was 60/hr rather than 15/hr.

Figure 3.

Rates of target-key responding for baseline and prefeeding test sessions (left panel) and for baseline (at 0 on the x-axis) and extinction test sessions in three-component multiple concurrent VI VI schedules (right panel; from Nevin et al., 1990, Exp. 2). During baseline training, the target response obtained 15 reinforcers/hr in Components A and B, and an alternative response obtained 45 reinforcers/hr concurrently in Component A only. In Component C, the target response obtained 60 reinforcers/hr). Note that the order of response rates for Components A and B reversed during the tests.

Rick and his students also conducted prefeeding and extinction tests of resistance to change. The effects of prefeeding 20 or 30 g in successive sessions are shown in the upper left panel of Figure 3, together with baseline data, averaged across pigeons. Target response rate in Component A was less reduced by prefeeding than in Component B, and there was a reversal of ordering at 30 g that was evident in the data of all three pigeons. The data for extinction are displayed similarly in the upper right panel, and again the reversal in the ordering of Components A and B was evident in the data of all three pigeons. For both prefeeding and extinction, resistance to change in Component C was greater than in Component B, as expected from the literature on resistance to change in multiple schedules. These results, which were published as Experiment 2 in Nevin et al. (1990), suggest that resistance to change of a target response depended on the total reinforcement in a component just as in the VI+VT component in Experiment 1 described above.

Clinical intervention and translation

In Experiment 1 of Nevin et al. (1990), the presentation of response-independent reinforcers in Component 2 parallels the use of noncontingent reinforcement (NCR) in applied analyses. In Experiment 2, reinforcement of an explicit alternative response in Component A parallels the use of differential reinforcement of alternative behavior (DRA) in applied analyses. Both NCR and DRA are common features of clinical interventions designed to reduce the frequency of a target problem response. For example, Iwata, Pace, Dorsey, Zarcone, Vollmer, Smith, Rodgers, Lerman, Shore, Mazaleski, Goh, Cowdery, Kalsher, McCosh, and Willis (1994) conducted functional analyses to identify reinforcers for self-injurious behavior (SIB) in people with developmental disabilities, and then arranged several sorts of reinforcer-based interventions including NCR. In 152 cases where NCR was employed, SIB decreased to below 10% of its pretreatment level in 84% of the interventions. The success rate was 83% for DRA. Comparable success rates were reported by Asmus, Ringdahl, Sellers, Call, Andelman, and Wacker (2004) for 138 participants who engaged in aggressive and disruptive behavior as well as SIB. These findings exemplify the power of reinforcement contingencies to reduce clinically significant problem behavior. But there is a worrisome implication of the data of Nevin et al. (1990) presented above: The success of NCR and DRA in reducing problem behavior may have the unintended consequence of making that behavior more resistant to further efforts to reduce or eliminate it because both DRA and NCR strengthen the contingency between stimuli and reinforcers.

Bud Mace and I discussed the VI+VT results (Figure 2) and their implications at a conference about a year before they were published, and within weeks, he and his colleagues at Lehigh and Rutgers implemented a close replication of the VI-only versus VI+VT procedure with two adult residents with mental retardation in a group home. The target response was sorting different colored utensils in alternation. Sorting both green and red items was reinforced according to a VI 1-min (60 reinforcers/hr) schedule, and in addition, response-independent reinforcers were presented according to a VT 30-s (120 reinforcers/hr) schedule while the participant was engaged in sorting one of the colors. Reinforcers were small cups of popcorn for one participant and coffee for the other. After 10-15 sessions, resistance to change was tested by turning on a video monitor with excerpts from an MTV program. The results, published by Mace, Lalli, Shea, Lalli, West, Roberts, and Nevin (1990, Part 2), are presented for individual subjects, pooled over two replications of video disruption, in Figure 4. The differences between baseline rates and the effects of disruptors in the two components are ordinally similar to but substantially clearer than the pigeons' results in Figures 2 and 3. DeLeon (2009) has reported similar effects of NCR on the persistence of problem behavior in people with developmental disabilities, so the result has substantial generality.

Figure 4.

Rates of responding on a sorting task by two adults with mental retardation (from Mace et al., 1990). Sorting utensils in multiple-schedule components defined by different utensil colors was reinforced according to a VI 1-min schedule (60 reinforcers/hr). In addition, response-independent reinforcers were given according to a VT 30-s schedule (120 reinforcers/hr) in one component. After baseline training (BL), sorting was disrupted by presenting a distractor (MTV). Note that the order of sorting rates for components with and without added VT reinforcers reversed during the tests.

Mace (this issue) has confirmed the results of Nevin et al. (1990, Experiment 2) with rats, demonstrating that concurrent reinforcement of an alternative response increased the resistance to extinction of a target response, and has obtained comparable results with problem behavior in children with developmental disabilities. He has also shown that these increases in resistance to extinction could be circumvented by reinforcing an alternative response in a distinctively different stimulus situation, so that alternative reinforcers do not enter into the Pavlovian contingency governing the persistence of problem behavior. The effectiveness of his procedure deserves experimental analysis in a reverse translation to the basic research laboratory.