Learning to Stop Responding

Abstract

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of “response error,” the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response—to what it is actually doing—as behavioral suppression is learned.

Learning to stop or inhibit a behavior may be just as important as learning to perform it to begin with, because the environments in which behaviors occur are typically variable and dynamic. But what do we really know about how we learn to stop responding? The present article addresses the question by discussing recent research that has studied three paradigms in which animals learn to stop performing an operant response: extinction, where the organism learns to stop responding when the behavior’s reinforcer is removed; punishment, where it learns to stop responding when the behavior now leads to an aversive consequence; and feature-negative learning, where it learns to stop responding when an inhibitory cue (or stop signal) indicates that an otherwise reinforced behavior will not be reinforced. In all of these paradigms, the animal must learn to withhold or inhibit responding if behavior is to adapt to the environment. But how is behavioral stoppage accomplished? What do animals learn, how do they learn it, and what might extinction, punishment, and feature-negative learning actually have in common?

Response inhibition in extinction

Extinction is probably the most extensively-studied of the learning-to-stop paradigms. In Pavlovian conditioning, responding elicited by a conditioned stimulus (CS) that has been associated with an unconditioned stimulus (US) diminishes when the CS is presented without the US. In operant learning, a response that has yielded a reinforcer similarly declines in rate or strength when the reinforcer is removed. One of the key insights about both Pavlovian and operant extinction discovered over the last few decades is that extinction learning is highly specific to the context in which it is learned (e.g., Bouton, Maren, & McNally, 2021). For example, in operant conditioning (the focus of this article), all three forms of the renewal effect have been observed (e.g., Bouton, Todd, Vurbic, & Winterbauer, 2011; Todd, Vurbic, & Bouton, 2014b; Vurbic, Gold, & Bouton, 2011). In ABA renewal, the extinguished response returns during testing when operant conditioning, extinction, and testing occur in Contexts A, B, and A, respectively. In ABC renewal, the contexts are A, B, and C. And in AAB renewal, the contexts are A, A, and B. The fact that renewal can occur in a non-conditioning context (i.e., B or C) tells us that extinction is more context-specific than the original operant learning. The phenomenon also tells us that extinction does not erase what was originally learned. And, as recently reviewed by Bouton (2019; see also Bouton et al., 2021), many different types of stimuli can play the role of context, including not only the typical experimental apparatus or operant chamber (e.g., Bouton et al., 2011; Todd et al., 2014a), but the animal’s hunger or food-deprivation state (Schepers & Bouton, 2017), stress state (Schepers & Bouton, 2019), preceding responses (Thrailkill, Trott, Zerr, & Bouton, 2016; see also Thomas, Thrailkill, Bouton, & Green, 2020), recent reinforcers (e.g., Baker, Steinwald, & Bouton, 1991; Bouton & Trask, 2016), and time (e.g., Bouton & Hendrix, 2011), to name a few. An extinguished response recovers or renews when it is tested outside the extinction context—defined in many ways.

However, a second thing we have learned about operant extinction is that it is specific to the response as well as the context. For example, in experiments conducted in our laboratory, Todd (2013) examined the three forms of renewal using a method in which the associative histories of the contexts were controlled. For instance, in a test of ABA renewal, rats were respectively reinforced for pressing a lever on a wall and pulling a chain dangling from the ceiling during intermixed sessions in two different contexts, A and B (see Table 1). (The boxes and the responses were fully counterbalanced, so that the rats learned to perform R1 in Context A and R2 in Context B.) In the next phase, the animals received extinction of both responses in the opposite context—R1 was extinguished in B and R2 was extinguished in A. Notice that R1 and R2 received complementary treatments, and that Contexts A and B were equally associated with conditioning, the reinforcer, and extinction. In final tests, both responses were tested in both contexts. There was a clear and specific renewal of R1 in Context A and R2 in Context B. During extinction, the animals had thus learned to inhibit each response in a specific context. Using the same kind of design, Todd (2013) also found response-specific renewal in ABC and AAB arrangements. Compatible results have been shown with discriminated operant situations (Todd, Vurbic, & Bouton, 2014b). And when Todd (2013) compared conditions in which renewal of a response was tested in a context where another response had been extinguished or not, there was no difference (Groups Ext-B vs. Ext-C, respectively, in Table 1, bottom). That is, there was zero measurable transfer of extinguishing one response on performing the other in the same context. In experiments on free operant extinction, the animal learns to stop making a specific response in a specific context.

Table 1.

Designs of two experiments from Todd (2013) that studied ABA renewal after extinction.

	Phase
Group	Acquisition	Extinction	Test
Experiment 1
	A: R1+	A: R2−	A: R1, R2
	B: R2+	B: R1−	B: R1, R2
Experiment 4
Ext-B	A: R1+	A: R2−	A: R2
	B: R2+	B: R1−	B: R2
	C: ---	C: ---
Ext-C	A: R1+	A: R2−	A: R2
	B: R2+	B: ---	B: R2
	C: ---	C: R1−

Note. A, B, and C refer to contexts; + refers to reinforcement; − refers to extinction. Experiment 1 was a within-subject design; thus, every subject received the treatments shown in each phase in an intermixed fashion. Experiment 4 contained two groups that received the treatments shown. In that experiment, there was equivalent renewal of R2 in Context B in the Test regardless of the differential treatment of Context B.

The lack of transfer of extinction from one response to another was extended in a series of experiments with discriminated operants, i.e., those in which the response was reinforced in the presence of a discriminative stimulus but not its absence (Bouton, Trask, Carranza-Jasso, 2016). In one of the experiments (Experiment 2; see Table 2), both lever pressing and chain pulling were reinforced when a 30-s tone stimulus (“S”) was presented (SR1+, SR2+). (SR1 and SR2 were trained in separate sessions; when training one R, the manipulandum for the other R was not available.) Then, one group was allowed to perform R1 in S without receiving reinforcement (Group SR1), another group (Group S) received the same extinction exposures to S− alone with the operant manipulanda removed, and a third (Group Control) was merely exposed to the apparatus during sessions of identical durations. We then tested SR1 and SR2. The results are summarized in Figure 1. Extinction of SR1 strongly suppressed R1, of course, but it had no discernible impact on R2. And perhaps just as interesting, Pavlovian extinction of S− had no impact on either response. The latter result surprised us because discriminative stimuli can become Pavlovian signals of the reinforcer (e.g., Colwill & Rescorla, 1986, 1988) and experiments on Pavlovian-instrumental transfer suggest that Pavlovian conditioning to cues can invigorate the operant response (e.g., Holmes, Marchand, & Coutureau, 2010). Our results, however, suggest that Pavlovian extinction of a discriminative stimulus is not sufficient to extinguish a discriminated operant, and that the animal must actually perform R1 in extinction to learn to stop responding. Furthermore, our results indicate that the content of this learning is specific to the response itself.

Table 2.

Designs of two experiments from Bouton, Trask, and Carrranza-Jasso (2016) that studied the response-specificity of discriminated operant extinction.

Phase
Group	Acquisition	Extinction	Test
Experiment 2
SR1	SR1+, SR2+	SR1−	SR1, SR2
S	SR1+, SR2+	S−	SR1, SR2
Control	SR1+, SR2+	---	SR1, SR2
Experiment 3
AR1−	AR1+, BR1+, CR2+, DR2+	AR1−	BR1, CR2
A−	AR1+, BR1+, CR2+, DR2+	A−	BR1, CR2
Control	AR1+, BR1+, CR2+, DR2+	---	BR1, CR2
Experiment 5
AR−	AR-O1, BR-O2	AR−	AR−, BR−
A−	AR-O1, BR-O2	A−	AR−, BR−
Control	AR-O1, BR-O2	---	AR−, BR−

Note. S, A, B, C, and D refer to different auditory and visual discriminative stimuli (counterbalanced). R1 and R2 refer to different responses (lever press and chain pull, counterbalanced). + refers to reinforcement; − refers to extinction. --- refers to no treatment. O1 and O2 refer to different outcomes (grain pellet and sucrose pellet). See text for full discussion and explanation.

Figure 1.

Test results of Bouton et al. (2016), Experiment 2

Note. As explained in the text, extinction of SR1 (Group SR1) suppressed performance of R1 in the discriminative stimulus—but not R2. And Pavlovian extinction of S (Group S) had no effect on either response. From Bouton et al. (2016). Reprinted by permission of the publisher.

Other experiments in the Bouton et al. (2016) series addressed related questions. For example, in Experiment 3 (see Table 2), animals learned to perform R1 in the presence of two 30-s stimuli “A” and “B” and R2 in the presence of two others “C” and “D.” We then extinguished AR1− (or A−, as before) and then tested BR1 and CR2. Notice that neither of the stimulus-response combinations that were tested had been extinguished directly. But extinction of AR1− profoundly weakened R1 in B and had no impact on R2 in C. And as before, Pavlovian extinction of A had no impact at all. Making the response was thus again necessary for extinction, and it inhibited responding in both the stimulus it had been extinguished in (A) and in a second one (B). A final experiment (Experiment 5) asked whether extinction inhibited the response directly, or signaled that the learned relationship between the response and its outcome was no longer in effect. Such hierarchical learning (i.e., S-[R-O]) would be similar to occasion setting (e.g., Fraser & Holland, 2019; Holland, 1992). In that experiment, the rats first learned to make one response (lever pressing) in stimuli A and B, but presses in A earned one type of food pellet (e.g., sucrose) and presses in B earned another one (e.g., a standard grain pellet). Extinction of AR− again suppressed responding in both A and B. Although there was a trend toward more responding in B, which we had not seen in other experiments in which responses had earned the same pellet, the trend was not statistically significant. The results thus suggested that extinction caused direct inhibition of the response, and not necessarily inhibition of knowledge of the response-outcome relation. It was as if the rat simply learned “stop lever pressing” instead of something cognitively richer, such as “stop the lever pressing that leads to sucrose.”

The importance of the response in extinction learning was further indicated by other experiments that explored the mechanism that might enable response-specific inhibition. Pavlovian conditioning theories since Rescorla and Wagner (1972) have emphasized what we now call “prediction error” and “error correction” in learning and extinction. In Pavlovian learning, when the US is weaker (or stronger) than what the CSs on a trial predict, the associative strengths of the CSs are updated on each trial (i.e., decremented or incremented, respectively) to correct the prediction error, with the correction being proportional to the size of the error. Informally speaking, the effect is to bring what the animal has learned to expect into alignment with what actually occurs on the trial. One phenomenon that supports the role of error correction in Pavlovian extinction is the “concurrent excitor” effect (e.g., Rescorla, 2000, 2006; Wagner, 1969). Here, two CSs are separately conditioned (A+ and B+), and then, in extinction, animals are given extinction trials with the two cues in compound (AB−) or with one of them alone (A−). The finding is that there is more decrement in responding to A if the animal receives extinction trials with AB− than A− alone. The explanation is that the associative strengths of A and B summate when they are combined, creating a greater prediction of the US, and consequently, greater prediction error and prediction error correction.

Importantly, the concurrent excitor effect has been reported in the extinction of discriminated operant learning as well (Janak, Bowers, & Corbit, 2012; Janak & Corbit, 2011; Kearns, Tunstall, & Weiss, 2012; Rescorla, 2000). For example, rats have first been reinforced for lever pressing in three different Ss (AR+, BR+, and CR+) and then received extinction trials of AR− and BCR−. Tests of responding in A and B then revealed less responding in B than A—suggesting that extinction was greater, once again, when it occurred with the combination of two cues. Although the results support the role of some form of error correction in discriminated operant learning, what error was actually corrected here? We know that animals can learn both Pavlovian (S-O) and operant (R-O) associations in discriminated operant learning (e.g., Bolles, 1972; Colwill & Rescorla, 1986, 1988). If the combined discriminative stimuli (B and C) have Pavlovian power, then perhaps the error correction is concerned with what the Ss predict, and the finding in operant studies is just another example of the Pavlovian concurrent excitor effect. The learning process could be correcting “stimulus error,” and adjusting what the stimuli predict. On the other hand, the results of the extinction phase of the experiments also revealed more lever pressing in the BC compound than in A alone. This fact could enable what Bouton, Thrailkill, Trask, and Alfaro (2020) have called “response error,” where the learning mechanism instead detects the difference between the level of responding in a stimulus and the level that the current reinforcer supports (presumably zero when there isn’t one). This discrepancy—response error—might cause a direct and proportional downward correction of the level of responding independently of stimulus error.

To separate the two possibilities, Bouton et al. (2020) set up a variation of the concurrent excitor experiment (Table 3). Rats first learned to make two different operant responses (lever pressing and chain pulling, as usual) in two discriminative stimuli: AR1+, BR1+, CR2+, and DR2+. Both the lever and chain were available on all trials. Then, after some initial extinction with each of the reinforced combinations (AR1−, BR1−, CR2−, and DR2−), all rats received extinction trials with the three-stimulus compound ABC. Both R1 and R2 were still available. We finally tested BR1 and CR2 in a reacquisition phase in which responding was reinforced. Notice that B and C both received extinction with the summed associative strengths of A, B, and C (e.g., Pearce, 1994; Rescorla & Wagner, 1972; Wagner, 2008). They therefore received the same stimulus error and stimulus error correction; if responding in B and C differed at the end, it could not be due to differences in stimulus error. But, because two of the cues in the ABC compound (A and B) promoted R1 and only one (C) promoted R2, we expected (and observed) more R1 than R2 responding during the ABC extinction trials. Thus, there was more response error for R1 than R2, and response error predicts that R1 should receive a proportionately bigger decrement than R2. And that is what we ultimately observed; there was less responding to BR1 than CR2 during the tests. The results are summarized in Figure 2. The findings thus isolated the importance of response error in discriminated operant extinction. An additional experiment also isolated response error in the related “overexpectation” paradigm (e.g., Lattal & Nakajima, 1998). In discriminated operant learning, the organism thus seems to correct its level of responding in proportion to the discrepancy between the response level and what the reinforcer actually supports.

Table 3.

Design of the concurrent-excitor experiment suggesting a role for response error in instrumental extinction (Bouton, Thrailkill, Trask, & Alfaro, 2020, Experiment 1).

Phase
Training	(Extinction)	Compound Extinction	Test
A: R1+R2−	A: R1−R2−
B: R1+R2−	B: R1−R2−	ABC: R1−R2−	B: R1+R2−
C: R2+R1−	C: R2−R1−		C: R2+R1−
D: R2+R1−	D: R2−R1−

Note. The experiment was within-subject, so all subjects received the treatments shown. A, B, C, and D = different discriminative stimuli (counterbalanced). R1 and R2 = lever press or chain pull responses (counterbalanced). + = reinforced; − = nonreinforced. During testing, R1 was higher in B than R2 was in C (Figure 2). See text for further explanation.

Figure 2.

Results of Bouton et al. (2020), Experiment 1

Note. Results of an experiment that confirmed a role for response error (as opposed to stimulus error) in discriminated operant extinction (Bouton et al.’s (2020) concurrent excitor experiment). See text and Table 3 for further explanation. Left, final single-element extinction trials with BR1 and CR2. Middle, responding during compound (ABC) extinction trials. Right, response rates during tests of BR1 and CR2. Lower responding on BR1 compared to CR2 indicates the effect of response error during the compound extinction phase. Reprinted by permission of the publisher.

Interestingly, the pattern may be the reverse in Pavlovian extinction learning, where stimulus error may be more important than response error. In a pigeon autoshaping experiment, Rescorla (2006) compared the effects of adding a Pavlovian excitor (e.g., a noise that was associated with food) and a positive occasion setter (e.g., a houselight that was associated with reinforcement of another keylight CS) on the extinction of pecking to a target keylight CS. In his arrangement, the excitor didn’t increase responding to the keylight in extinction, although it presumably increased the bird’s Pavlovian expectation of food. Conversely, the occasion-setter enhanced responding to the target CS, although perhaps not the bird’s expectation of the US. When the target keylight was finally tested, the excitor, but not the occasion setter, had augmented extinction when compared to the effect of extinction with the keylight alone. Thus, stimulus error was more important than response error in a Pavlovian example of the concurrent excitor effect.

To summarize, when animals learn to stop performing an operant response in extinction, what they learn is specific to the response as well as the context. In a preliminary way, our results further suggest that the response is weakened because of the correction of response error: The learning mechanism adjusts the level of responding so that it proportionately corrects the discrepancy between its current level and what the reinforcer supports. We should note that Rescorla has reported compatible results in operant extinction. Some of his results likewise suggest that instrumental extinction learning can be specific to the response (Rescorla, 1993, 1997), and more extinction learning accrues when the extinguishing response is emitted at a higher level (e.g., Rescorla, 1997)—consistent with a role for response error.

Response inhibition in punishment

Punishment is of course a second reason to learn to stop responding. In punishment, unlike extinction, the response remains reinforced, but now it also earns an aversive stimulus such as footshock (e.g., Bouton & Schepers, 2015; Broomer & Bouton, 2023; Estes, 1944; Krasnova et al., 2014; Marchant et al., 2013; Panlilio et al., 2003). Responding therefore declines. Recent thinking about punishment has emphasized that it is an especially relevant way to conceptualize learning to stop responding in many clinical settings. For example, because reinforcers are always available for drug-taking, gambling, and over-eating, anticipation of their negative consequences is arguably more plausible than extinction as a mechanism for stopping these behaviors. Patients attempting abstinence from such behaviors are indeed often motivated by avoidance of their adverse outcomes (e.g., Downey et al., 2001; Marchant et al., 2019; Riedel et al., 2002). However, although the procedural differences between punishment and extinction are undoubtedly important, recent evidence suggests that they share important characteristics (see Bouton et al., 2021 for a review).

For one thing, punishment learning, like extinction learning, is highly specific to its context: Renewal has recently been observed in several experiments when the context is changed after punishment (e.g., Bouton & Schepers, 2015; Broomer & Bouton, 2023; Marchant et al., 2013). For example, Bouton and Schepers (2015) reinforced rats for lever pressing in one context (A), then punished it in a separate context (B). During punishment, responding was still reinforced with food pellets, but now also earned an occasional footshock on a variable-interval 90-s schedule. As a result, the rats quickly learned to stop responding in Context B. But responding renewed substantially on return to Context A (an ABA renewal effect). In a similar experiment (Bouton & Schepers, 2015; Experiment 2), responding that was punished in Context B also renewed when it was tested in a third, neutral Context C (ABC renewal). In both cases, no renewal was observed in yoked-control subjects that received the same shocks as punished rats, but noncontingently during the punishment phase. (There was little behavioral suppression at all there.) Thus, the punished response was not merely renewing in Contexts A and C because simple fear conditioning of Context B suppressed responding there. The observations of renewal after punishment obviously parallel results in operant extinction (e.g., Bouton et al., 2011; Bouton et al., 2021) and suggest that punishment learning, like extinction, involves learning about the context.

Also like extinction, punishment learning can be remarkably specific to the response. In an adaptation of Todd’s (2013) extinction procedure described earlier, Bouton and Schepers (2015, Experiment 3; see Table 4) trained R1 in Context A and R2 in Context B, then punished R1 in Context B and R2 in Context A. (The responses were both reinforced and punished in the new context.) Once again, after suppression was complete, R1 was shown to renew in Context A and R2 renewed in Context B. As in extinction, rats had thus learned to inhibit a specific response in a specific context. Interestingly, the renewal of each specific response was obtained regardless of whether the two responses were tested simultaneously or separately (in separate sessions); the results with the simultaneous test indicated that punishment can affect choice in a context- and response-specific way, as had been shown previously in extinction (Todd, 2013). Bouton and Schepers further noted that the context- and response-specificity of punishment challenges the classic theories of punishment. For instance, response-specific inhibition is not consistent with the simple conditioning of an emotional response to the punishment context (e.g., Estes, 1969), which would have affected both responses equally. Similarly, any incompatible response learned during punishment (e.g., Guthrie, 1935; Solomon, 1964) would likewise compete equally with R1 and R2. Instead, Bouton and Schepers’s results are consistent with the idea that punishment can involve direct learning about the response itself (e.g., Bolles, Holtz, Dunn, & Hill, 1980), and we suggest that it could also involve the response error correction process that we have tentatively identified in extinction (Bouton et al., 2020): In punishment, the level of responding may adjust downward to reduce the discrepancy between the current level of responding and what the current consequence supports. The results clearly establish a parallel between punishment and extinction and suggest that they follow similar rules.

Table 4.

Design of experiment demonstrating ABA renewal after punishment (Bouton & Schepers, 2015).

Phase
Acquisition	Punishment	Test
A: R1-food	A: R2-food + shock	A: R1, R2
B: R2-food	B: R1-food + shock	B: R1, R2

Note. A and B refer to contexts. The experimental design was within-subject; thus, every subject received the treatments shown in each phase in an intermixed fashion.

The apparent parallel between extinction and punishment may also include negative punishment, where responding is suppressed by withholding a reinforcer that is otherwise scheduled to occur (as in omission or differential reinforcement of other behavior [DRO] procedures). For example, Nakajima et al. (2002) reported strong ABA renewal of an operant response after it was suppressed by an omission contingency. More recently, Rey, Thrailkill, Goldberg, and Bouton (2020) compared the effects of omission and extinction directly. After training rats to lever press in Context A, the response was suppressed in Context B with either extinction or an omission procedure. After the treatments had produced similar asymptotic levels of response suppression, the response was tested back in Context A. Similar amounts of ABA renewal were observed. Thus, negative punishment was no less sensitive than extinction to renewal. Interestingly, however, the same was not true for reinstatement, the “relapse” effect that can be created by introducing noncontingent reinforcers after extinction (e.g., Franks & Lattal, 1976; Reid, 1958; Rescorla & Skucy, 1969): Rats that had undergone extinction showed greater reinstatement of responding when noncontingent food pellets were delivered than rats that had undergone omission training, where the reinstating pellets had no effect (see also Uhl & Garcia, 1969). This result can be understood by noting that the delivery of food pellets had been a salient part of the “context” in which rats had learned to inhibit responding during omission training. Thus, while the results further support a common contextual account of punishment and extinction, they remind us that the presence of the reinforcer during punishment can create differences in the effects of the two procedures.

More recent work by Broomer and Bouton (2023) makes this point in another way. We systematically compared extinction and positive punishment. When we studied ABA renewal following either shock-induced punishment or extinction, the punished and extinguished groups exhibited essentially identical degrees of renewal (Figure 3). Interestingly, the punished and extinguished responses also both showed spontaneous recovery over the 24-hour interval between consecutive response-elimination sessions, and in another experiment, similar levels of spontaneous recovery after a 7-day retention interval. These results can be seen from the perspective that spontaneous recovery is a form of renewal that occurs with a change of temporal context (see Bouton et al., 2021 for a review). More generally, they suggest that the effects of punishment and extinction are both vulnerable to changes in physical and temporal context, despite the intuition that punishment may be a stronger or more effective means of suppressing the response.

Figure 3.

Results of Broomer and Bouton (2023), Experiment 1a

Note. Lever-press data for each stage of training and test in Broomer and Bouton (2023), Experiment 1a. Left, acquisition of lever pressing in Context A. Middle, effects of the response elimination treatments in Context B. Full recovery of Yoked group responding indicates that response suppression in the Punished group was due to the response-shock contingency. At test (right), Punished and Extinguished groups exhibited nearly identical degrees of ABA renewal. From Broomer and Bouton, 2023. Reprinted by permission of the publisher.

Of course, as we found with negative punishment, similar renewal and spontaneous-recovery patterns do not mean that positive punishment and extinction are exactly the same. Broomer and Bouton (2023) also found that when an operant response was reinforced again following extinction or punishment, the extinguished response was reacquired rapidly (and much faster than they when the rats had originally learned the response), while the punished response was reacquired slowly—at about the rate at which the rats had originally learned the response. The difference could superficially suggest that punishment produces stronger response suppression than extinction. However, the similar renewal and spontaneous recovery following punishment and extinction clearly challenges this interpretation. Instead, the result was again predicted by a contextual analysis of punishment: Unlike in extinction, reinforcer presentations are part of the “context” present during positive punishment at least initially (before the response is suppressed), and this permits more transfer of behavioral inhibition from the response elimination to the reacquisition phase. Thus, the difference in the rate of reacquisition is consistent with the notion that punishment and extinction are similarly context-dependent but are learned in different reinforcer contexts.

Ultimately, punishment follows rules that are similar to those of extinction: It can be specific to both the context and the response. These similarities exist despite important procedural differences, namely the availability of the reinforcer during response suppression and the introduction of an aversive and potentially emotion-arousing stimulus as a consequence. Although the latter differences can be consequential in certain situations (i.e., reacquisition), they do not indicate that punishment and extinction follow different rules, or that punishment necessarily produces a stronger or more durable form of behavioral suppression. They both result in the animal learning to stop making the specific response in a specific context.

Response inhibition in feature-negative learning

Historically, inhibition of behavior has arguably been studied most intensively in Pavlovian conditioning. In that literature, one of the best ways to achieve inhibition is the so-called “conditioned inhibition” or “feature negative” (FN) procedure. In this arrangement, a CS (A) signals a US when it is presented on its own, but not when it is combined with a “feature” stimulus (B) (i.e., intermixed A+ and AB− trials). As intermixed A+ and AB− trials continue, the animal learns to respond to A, but stops making the response when B is combined with A. The discrimination is conventionally thought to work because B acquires inhibition, which can be confirmed with summation tests (in which B is combined with a separately-conditioned excitor, e.g., C, and shown to inhibit responding to it) and retardation tests (where B alone is now paired with a US and is shown to acquire the ability to evoke a response only slowly) (e.g., Rescorla, 1969). Notice that inhibition to B “transfers” and affects responding to another CS (C) in the summation test. And it inhibits responding to C even if C happens to excite a different response than the one excited by A (Rescorla & Holland, 1977). That result indicates that Pavlovian inhibition to B is not inhibition of a specific response. In contrast, B does not inhibit responding to C if A and C have been associated with different appetitive USs (Delamater et al., 2003; Kruse et al., 1983; Laurent & Balleine, 2015; but see Kemp & Corbit, 2021). Therefore, Pavlovian inhibitors, like B in the FN design, are thought to inhibit Pavlovian responding by suppressing a memory or memory node that represents the US.

The feature-negative paradigm has also been studied in operant conditioning, and it therefore provides a third place where we can study how animals learn to stop operant responding. In the operant case, a response like lever pressing is reinforced in A (AR+) but not when A is combined with B (ABR−). The animal again learns to respond in A and to stop responding in AB. B becomes an inhibitor or a stop signal. But what is B actually inhibiting? We know that B can transfer and inhibit the same operant response occasioned by a different stimulus (e.g., Holland, 1991). And surprisingly, given all the evidence above that inhibition learned in operant extinction and punishment can be highly response-specific, there is evidence that B can also inhibit a different, separately-trained operant response (Gutman & Maier, 1978; Hearst & Peterson, 1973; Holland & Coldwell, 1993; but see Bonardi, 1989). Importantly, though, same-response and different-response inhibition had not been compared in a way that tested the completeness of the transfer to a new response. Given all the evidence of response-specificity in extinction and punishment learning, could different-response inhibition really be as strong as same-response inhibition? Steinfeld and Bouton (2022) therefore designed several experiments that allowed a direct comparison of same-response and cross-response inhibition in the FN design.

Our method was to train lever pressing and chain pulling in separate but parallel FN discriminations (at left in Table 5). In sessions where R1 was available, rats received AR1+ and ABR1− trials, and in sessions where R2 was available, rats received CR2+ and CDR2− trials. The question was, would B’s ability to inhibit R1 transfer and influence R2 (and would D’s ability to inhibit R2 transfer and inhibit R1)? We tested this several ways. In a novel kind of test (an “inhibitor test;” see Table 5), after FN training, R1 and R2 were freely reinforced on a random-interval schedule to bring responding above the low level that was otherwise evident between trials. Then we presented B and D alone while the rats were engaged in making either response. R1 and R2 were tested in separate sessions. Would B only inhibit R1 and D only R2? In a subsequent summation test (a “compound test”), after reminder sessions on the two FN discriminations, we compared AR1, ABR1, and ADR1 (as well as CR2, CBR2, and CDR2). Again, would there be more inhibition of R1 by B (on ABR1 trials) than D (on ADR1 trials), and more inhibition of R2 by D (on CDR2 trials) than B (on CBR2 trials)? Remarkably, in several experiments, the results were consistent across the two tests: Inhibition created by B and D transferred beautifully and completely across the two responses (see Figure 4). That is, B and D inhibited either R1 or R2; there was no evidence of a loss of inhibition when we tested either with the different (i.e., nontrained) response. In contrast, A and C, the stimuli trained to set the occasion for R1 and R2, respectively, selectively occasioned R1 and R2 in an “excitor test” (although there was significant transfer between them). Inhibition to B and D in the FN method was completely response-general, though the excitatory effect of A and C was not. In other words, the FN method had taught animals simply to stop responding, but not to stop making a specific response.

Table 5.

Design of an experiment demonstrating the response-generality of inhibition learned in an operant feature-negative discrimination (Steinfeld & Bouton, 2022, Experiment 1).

Phase
Acquisition	Inhibitor Test	Excitor Test	Compound Test
AR1+, ABR1−	(R1+) R1−, BR1−, DR1−	R1−, AR1−, CR1−	AR1+, ABR1−, ADR1−
CR2+, CDR2−	(R2+) R2−, BR2−, DR2−	R2−, AR2−, CR2−	CR2+, CBR2−, CDR2−

Note. This experiment was within-subject, so all subjects received the treatments shown. + = reinforced; − = not reinforced; A, B, C, and D are different 30-s cues (counterbalanced); R1 and R2 are different responses (lever press or chain pull, counterbalanced. Animals received reminder sessions of compound training (“acquisition”) after each of the different tests beginning with the Inhibitor Test (the reminder sessions are not shown).

Figure 4.

Test results of Steinfeld and Bouton (2022), Experiment 1

Note. Test data from Steinfeld and Bouton (2022), Experiment 1. A) In the inhibitor test, feature-negative inhibitors B and D were equally effective in suppressing the responses with which they were trained (i.e., BR1, DR2) as well as the “wrong” responses (i.e., BR2, DR1). B) In the compound test, B and D similarly suppressed both responses when presented in compound with the respective excitors (i.e., ABR1, ADR1, CBR2, CDR1). C) In the excitor test, excitatory cues A and C selectively occasioned R1 and R2, respectively, albeit with significant cross-response transfer. See text and Table 5 for further explanation.

The results were surprising given the research reviewed above that suggested so much response-specificity of inhibition in extinction and punishment. Perhaps inhibition by B and D was response-general because they were uniquely functioning as Pavlovian inhibitors, i.e., signals that no reinforcer would occur. Such knowledge could influence both responses; recall that the effects of Pavlovian inhibitors transfer across different Pavlovian conditioned responses (Rescorla & Holland, 1977). Was that the explanation? The evidence we collected mainly suggested “no.” First, systematic observation of the rats’ behavior during training revealed no clear Pavlovian inhibitory conditioned response. We were especially interested in possible movement away from the inhibitors’ location (e.g., as demonstrated, for example, by Wasserman et al., 1974), which could interfere equally with R1 and R2. But our systematic observations uncovered no evidence of this. The animals sat in the chamber (with some movement) until presentations of A or C alone caused them to get up and begin lever pressing or chain pulling, and presenting the inhibitors mainly prevented this. Also, the strength of the inhibition did not depend on whether the lights or speakers providing B and D were located on the same wall as the lever and chain or on the opposite wall. Second, perfect cross-response inhibition was still evident when AR1 earned one reinforcer (O1) and CR2 earned a different one (O2; sucrose or grain pellets, counterbalanced). Recall that Pavlovian inhibition does not appear to transfer across reinforcers (Delamater et al., 2003; Kruse et al., 1983; Laurent & Balleine, 2015; but see Kemp & Corbit, 2021). Thus, although the FN procedure could have allowed the rats to learn that B and D signaled that O1 or O2 was not coming, it is difficult to see how such inhibitory signaling could transfer and produce inhibition of a different response associated with a different reinforcer.

One difference between the FN experiments, where inhibition transferred so well across responses, and our previous extinction experiments, where the inhibition did not transfer, was that cross-response inhibition in the FN experiments was tested with a response that had been inhibited by a stimulus (either B or D) throughout the initial training. Thus, if the rat had been able to learn that R1 and R2 were “inhibitable” when B and D respectively led responding to stop, then “inhibitability” of the target might have encouraged the transfer of inhibition. Inhibitability did make a difference: When we compared cross-response inhibition following the usual AR1+, ABR1−, CR2+, CDR2− procedure and one in which R2 was never inhibited (e.g., AR1+, ABR1−, CR2+ only), there was significantly less cross-response inhibition in the latter, though a significant amount of it remained. The effect of “inhibitability” fits with evidence from the occasion setting literature suggesting that negative occasion setters modulate responding to stimuli that have a history of inhibition (e.g., Swartzentruber & Rescorla, 1994). Holland (e.g., 1992) has argued that such results suggest that occasion setters transfer and operate within a conditional memory system; Rescorla (1985) and Bouton and Nelson (1994) have suggested that inhibitors can modulate inhibition to the target. But surprisingly, when we went on to ask whether making the transfer target inhibitable created response-general inhibition in an extinction design, the effect of extinction was still very response-specific. That is, after training with AR1+ and AR2+, the extinction of AR− still had little effect on AR2 even if we had made R2 “inhibitable” with AR2+, ABR2− training. Operant extinction and punishment appear response-specific, while the inhibition learned in operant FN learning is not.

The inhibition we studied in operant FN learning thus appears to involve a second mechanism for learning to stop responding: B and D produce response-general inhibition, whereas extinction and punishment involve response-specific inhibition. It is as if B and D in the FN case tell the animal simply to “stop responding” instead of “stop making that specific response.” (And the animal is most likely to stop the response if it has been made inhibitable prior to testing the signal.) Our research on extinction, punishment, and FN learning has thus uncovered evidence of both response-specific and response-general mechanisms allowing an animal to learn to stop making an operant response.

Response-specific inhibition: A possible role for attention to the response

The response-general nature of inhibition learned in the FN paradigm has led to additional (as-yet unpublished) work (Bouton, Broomer, Frazee, & Steinfeld, in progress). There is an obvious “pureness” to the inhibition controlled by B and D in the FN paradigm: B and D are exclusively inhibitors, and have never been anything else. In contrast, stimuli associated with extinguished or punished operants have had a history of both reinforcement and then behavioral suppression. One consequence is that the inhibition learned in extinction or punishment might simply be weaker than that learned in FN learning. Is response-general inhibition just stronger inhibition? Our additional research suggests that the answer may be “no.” In one experiment, we compared the response-specificity of inhibition that developed after 3, 6, or 12 days of daily FN training with the usual Steinfeld-Bouton procedure (at left in Table 5). (Three days was the lowest amount of training that gave B and D an ability to suppress responding, and 12 days was the standard amount used in the Steinfeld and Bouton, 2022, experiments.) In our usual tests, we found that inhibition after three days of FN training was weaker, but apparently just as response-general, as it was after 12. Thus, inhibition arising from the FN method does not seem response-general merely because it is strong.

A second consequence of B’s unambiguous signaling of nonreinforcement is that it might attract considerable attention. There are a number of reasons to think that attention may play a role in the extinction, punishment, and FN learning procedures. For one thing, recent research in other laboratories suggests that when a Pavlovian stimulus’s meaning changes at the start of extinction, there is a boost in attention to other cues. For example, in experiments with human participants, when there is an abrupt discrimination reversal so that a reinforced S is nonreinforced and a nonreinforced S is reinforced, the participant directs his or her attention to irrelevant cues in the background, as measured by eye-gaze (Torrents-Rodas, Koenig, Uengoer, & Lachnit, 2021) or dot-probe methods (Vadillo, Orgaz, Luque, & Nelson, 2016). Relatedly, Rosas and colleagues (e.g., Rosas & Callejas-Aguilera, 2006) have long argued and produced evidence that extinction leads the participant to increase attention to and learn about the background context. Could an attention boost that occurs at the start of instrumental extinction also boost attention to the animal’s response, making inhibition learning in extinction response-specific as well as context-specific? Interestingly, in the FN design, as emphasized above, stimulus B is introduced and perfectly predicts nonreinforcement when nonreinforced trials begin. If attention is directed toward the most predictive event or stimulus, while attention to other cues declines, as an attentional theory of Pavlovian learning like that of Mackintosh (1975) generally proposes (see also, e.g., Pearce & Mackintosh, 2010), then B could capture more attention than the response. The attention that is boosted during nonreinforcement in the FN design might therefore go to B, rather than the response, encouraging less learning about the specific response.

One way to test this idea would be to make B less predictive of nonreinforcement in the FN procedure. If it were trained to set the occasion for the response (rather than inhibit it) before FN training begins, it would be a less reliable predictor of nonreinforcement, and thus less likely to win the attention competition. Would this allow more attention to the response and consequent response-specificity of inhibition? In a recent experiment, we tested this idea. We first trained B to set the occasion for R1 (BR1+) and D to set the occasion for R2 (DR2+). Then we began training our usual parallel FN discriminations (AR1+, ABR1−, CR2+, CDR2−). At this point, B and D now played inhibitory roles, but inhibition was slow to develop because of their prior training as excitors. In a compound test, there was less inhibition when the inhibitors were tested with the other response (ADR1 and CBR2) than with the same response (ABR1 and CDR2). That is, we found response-specific inhibition. Recall that Steinfeld and Bouton (2022) repeatedly observed no difference here (e.g., Figure 4). Thus, when we first gave B and D an excitatory meaning and then changed it to an inhibitory one, inhibition in the FN procedure became more specific to the response.

This appeal to the amount of attention paid to a behavior is consistent with other research in our laboratory on goal-directed actions and habits (Bouton, 2021). That research suggests that attention to an instrumental response might decline (and the response become a habit) when the reinforcer becomes well-predicted by the discriminative stimulus (Thrailkill, Trask, Vidal, Alcalá, & Bouton, 2018; Thrailkill, Michaud, & Bouton, 2021). We have suggested that the mechanism is compatible with attention rules proposed by theories of Pavlovian conditioning (e.g., Pearce & Hall, 1980; Pearce & Mackintosh, 2010; see also Mackintosh, 1975; Bouton, 2021, provides more complete discussion). Conversely, attention to a habitual response might increase (and return the behavior to goal-directed status) when the outcome is made surprising (Bouton, Broomer, Rey, & Thrailkill, 2020; Trask, Shipman, Green, & Bouton, 2020). Attention to the behavior might be required for learning something new about the behavior. There is thus room for more research on the role of attention to the response when we think about actions, habits, and the response-specificity of inhibition.

What then causes the response-general inhibition evident in the FN discrimination? Steinfeld and Bouton (2022) noted that animals are thought to encode both the sensory and the motivational attributes of the reinforcer (e.g., Konorski, 1967; Wagner & Brandon, 1989). Presentation of an inhibitory Pavlovian cue might therefore suppress the representation of either the sensory or motivational attributes (or both). As reviewed above, B and D transfer and inhibit the alternative response even when the responses earn different food-pellet reinforcers (Steinfeld & Bouton, 2022, Experiment 3). Thus, their effect could not be to inhibit common sensory attributes. But both reinforcers in that experiment were food pellets. Perhaps B and D inhibited the representation of their common motivational value, demotivating both R1 and R2 in the compound and inhibitor tests. By itself, this mechanism does not explain the additional importance of the response’s inhibitability in allowing response-general inhibition. But it might begin to solve the problem. There are a number of learning processes that may come into play when organisms learn to stop responding.