Facets of Pavlovian and operant extinction

Abstract

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.

The title of this paper encompasses two relations: that between Pavlovian and operant conditioning and the relation of both of these to extinction. The former is, of course, longstanding in the study of learning, incorporating different empirical and theoretical perspectives (e.g., Blackman, 1977; Mackintosh, 1983; Rescorla & Solomon, 1967). Both have supported rich empirical research programs and equally fruitful developments in the theoretical understanding and practical applications of learning. Central to all of these activities has been the experimental analysis of extinction.

Theoretical perspectives on Pavlovian-operant relations include the position that the nominally different types of conditioning are, in fact, indistinguishable (e.g., Hearst, 1976), to the assignment of more relative importance to one or the other (e.g., Skinner, 1938) to, more commonly, interactive positions identifying important roles for both (e.g., Rescorla & Solomon, 1967; Mowrer, 1960; Blackman, 1970). Similarities abound in terms of behavioral effects and controlling processes. Both Pavlovian and operant situations are comprised of relations among stimuli, responses, and outcomes. The relative importance of various relations among these elements often defines Pavlovian (stimulus-outcome) and operant (response-outcome) conditioning, but relations with responses are important in Pavlovian conditioning and relations with stimuli are important in operant conditioning (e.g., Blackman, 1977; Nevin, & Grace, 2000; Crombag, Galarce, & Holland, 2008).

Pavlovian and operant extinction are both microcosms of broader discussions of the Pavlovian-operant relation and, independent of such discussions, a rich source of empirical and theoretical analyses of learned behavior. On the one hand, despite the procedural differences that define the two preparations, there are commonalities in definitions of extinction, methods of analysis, and empirical findings. On the other hand, a host of similar findings have yielded different interpretations and theories that have contributed to not only the understanding of extinction, but also to more general issues in the psychology and neurobiology of learning.

In this review, we consider some of the many facets of Pavlovian and operant extinction. We begin with a description of the ways in which the term “extinction” has been used in the literature. We then describe some issues to consider in the designs and analyses of extinction experiments. Next, we examine the role that theory has played in understanding extinction. Finally, we end with considerations of some applications of Pavlovian and operant extinction research.

Terminology

The term “extinction” is used in both Pavlovian and operant preparations to describe procedures and the behavioral outcomes of those procedures, as well as a behavioral process or mechanism that underlies these outcomes. Extinction is most often procedurally defined as the omission of previously delivered unconditioned stimuli or reinforcers; however, it also has been defined as the absence of a contingency between response and reinforcer (e.g., Baker, 1990; Premack, 1965; Rescorla and Skucy, 1969) or between conditional and unconditional stimulus (e.g., Delamater, 1995; Lindblom & Jenkins, 1981). Statements like “extinction consisted of 10 presentations of the conditional stimulus (CS) without the unconditional stimulus (US)” or “extinction was implemented by removing reinforcer access” illustrate this usage.

At a functional, behavioral level, extinction describes the decreases in responding from higher levels observed prior to extinction to lower levels following implementation of one of the aforementioned operations. Statements like ““responding extinguished by the tenth trial” or “responding was extinguished over several sessions” illustrate this usage. Even this well-established outcome, however, is subject to certain qualifications. First, the time course and functional extent of response elimination depend on how the conditions of extinction differ from the conditions present before extinction (during initial acquisition and maintenance; e.g., Gibbon, et al. 1980; Kimble, 1961; Nevin & Grace, 2000). Second, even though extinguished responding may be completely eliminated, there are a variety of conditions that will cause at least some of the responding to return (see Delamater, 2004; K.A. Lattal, St. Peter, & Escobar, in press). Also with respect to outcomes, concurrent with reducing the learned response that is the target of the extinction contingency, extinction may cause other responses to be generated (see K. A. Lattal, et al., in press, for a review). Unless reinforced, however, such generative behavior is transient, disappearing in the absence of its reinforcement.

At a theoretical level, extinction can refer to or imply a behavioral mechanism or process. Statements like “inhibitory extinction processes do not erase the original learning” illustrate this usage. Commonalities of behavioral effects suggest related, or at least overlapping, processes in Pavlovian and operant extinction. As will be developed in subsequent sections of this review, different theories of extinction have focused on external (observable) and internal (inferred) processes. The former tend to be the focus in behavior-analytic theories of operant extinction and the latter the focus of associative theories of Pavlovian and operant extinction. That said, however, it is an oversimplification to associate a particular theoretical perspective exclusively with one type of behavioral approach (e.g., Nevin & Grace, 2000; Rescorla, 1987; Woodruff, Conner, Gamzu, & Williams, 1977).

Experimental Design

Despite the procedural differences in Pavlovian and operant preparations, many of the methodological challenges in studying extinction are common to the two approaches. These challenges are of two types: those related to procedures and those related to measurement.

Procedures

Comparisons of effects of variables during extinction in Pavlovian or operant preparations can be made either across different subjects or by exposing the same subject to each condition sequentially. The former, between-subject, designs are more common with Pavlovian preparations simply because many such studies focus on rapidly occurring changes during initial acquisition or extinction. The latter, within-subject, designs are used often in operant preparations where resistance to extinction of responding maintained at the steady state often is of interest. Operant approaches also often involve repeated returns to baseline between phases, whereas Pavlovian approaches typically do not. The preparations are not defined, however, by their uses of between- or within-subject designs, as there are many examples of operant extinction studied using between-subject designs (e.g., Rescorla & Skucy, 1969) and Pavlovian extinction studied using within-subject designs (e.g., Gottlieb & Rescorla, 2010).

The advantages and limitations of each category of design are well documented (Rescorla & Holland, 1976; Sidman, 1960). Between-subject designs present challenges in matching all of the experimental conditions in different groups outside of the manipulation of interest. Among the many variables that may differ between groups, for example, might be total exposure to a context and number or rate of presentations of an unconditioned stimulus or reinforcer. These differences can result in behavioral differences that may be misattributed to the manipulation of interest. For example, after Pavlovian fear conditioning, in which a tone is paired with a shock, nonreinforced presentations of the tone will extinguish tone-evoked responses, compared to a No Extinction group that remained in its home cage during extinction. However, the No Extinction group also will have received less exposure to the apparatus, fewer experiences with the tone, and less handling by the experimenter. Any of these additional differences in treatment during the extinction manipulation may cause the group differences observed during a test.

Within-subjects approaches are better able to match these differences in treatment, but they present their own challenges. For example, in operant approaches, baseline responding often needs to be established and reestablished between successive extinction periods. In operant and Pavlovian preparations, the use of multiple stimuli or responses will result in the treatment in the presence of one stimulus influencing responding in the presence of the other (e.g., response induction, Hemmes & Eckerman, 1972). Thus, when extinction is imposed following differential training in the presence of two stimuli, it has to be implemented following one of the conditions first. As a result, when the second condition is presented, the subject has had at least some history of extinction. Even though the extinction responding may be under stimulus control, the within-subject comparisons of extinction as a function of the different parameters in effect during training may be tainted. Counterbalancing the treatment of the stimulus that is presented first in extinction helps with this problem: if differences in extinction occur independently of which stimulus is first presented, then it can be concluded that the differences in extinction are due to the independent variables and not simply an order effect. This ideal outcome, however, is not always the case and emphasizes the importance of including other designs that manipulate the variable of interest during extinction in different ways.

Measurement

In addition to the type of general procedures (between or within-subjects), Pavlovian and operant experiments are complicated by multiple measurement considerations in how to evaluate the response loss that occurs during extinction. In discrete-trial procedures (both Pavlovian and operant), a typical extinction curve is plotted as a response measure (e.g., response rate, probability, or latency) as a function of the number of extinction trials. In discrete trial and free-operant procedures, responding also has been recorded as a cumulative frequency graph (e.g., Leslie, Shaw, Gregg, McCormick, Reynolds, & Dawson, 2005; Skinner, 1938). These measures reveal effects on extinction as a function of the experience specifically during extinction.

Other measures reveal effects by examining how behavior changes during extinction as a function of the relative differences between the response measure before and during extinction. For example, in some experiments extinction has been plotted as a function of the number of omitted reinforcers during extinction that might have been expected based on the conditions of acquisition (e.g., examining behavior during every fourth trial during extinction following a partial reinforcement schedule in which a reinforcer occurs on 25% of trials; see Gibbon, et al. 1980). These measures, as well as measures of amount of time or number of trials to reach a specified period in which the response is absent, have revealed effects that may be opposite to those effects found when responses are plotted as a function of number of extinction trials (e.g., Gallistel & Gibbon, 2000, Nevin & Grace, 2005). Thus, the dependent variable during extinction may affect the interpretation of the long-term effects of manipulations that occurred prior to extinction.

With a given dependent variable, effects on rate of extinction often are inferred from differences in the shape of behavioral extinction functions. These approaches are useful for quantitative modeling of how alterations in environmental contingencies affect steady-state performance and for evaluating how a manipulation affects the rate of extinction from a common behavioral starting point at the outset of extinction. Assessing how extinction occurs from different starting levels of response is more difficult. As Nevin, Smith and Roberts (1987) noted, “downward variation is more limited for the performance with the lower baseline, biasing the comparison of differences in favor of the conclusion that the performance with the higher baseline rate is more resistant to change” (p. 31). Conversely, during early extinction trials, more absolute response loss may occur when behavior starts at a higher point, due to the larger discrepancy (or prediction error) between the signaled and delivered reinforcer (e.g., Rescorla & Wagner, 1972). Thus, if the criterion for demonstrating extinction is rate of response decrease or loss, a higher starting point may satisfy that criterion faster. If the criterion is number of trials to reach a point of low responding, then the lower starting point may satisfy that criterion faster.

One potential solution to the problem of different levels at the beginning or end of extinction is to normalize response rates relative to responding during the pre-extinction condition (see Nevin, et al. [1987] for a discussion of the details for conducting such analyses). This is a commonly used tool that is helpful for drawing conclusions about how performance during extinction changes as a function of levels of pre-extinction performance. However, if a goal is to compare the efficacy of a given manipulation during extinction, large differences in behavior prior to extinction make even normalized comparisons challenging because one still needs to make assumptions about how relatively high and low levels of responses map onto underlying learning processes. If theories can be developed to make explicit predictions about deviations from different points on a scale, then this approach can be justified. A different solution, when possible, is to address the problem experimentally by equating performance prior to extinction manipulations (e.g., K.M. Lattal, 1999; Stafford & K.M. Lattal, 2009).

These issues of measurement during extinction all warrant consideration when evaluating how behavior maintained by either Pavlovian or operant preparations changes when extinction contingencies are in effect (i.e., during extinction). Another measurement issue occurs when attempting to assess the long-term impact of those extinction contingencies (i.e., after extinction). The strongest conclusions about the long-term effects of an extinction manipulation come from post-extinction tests, during which the effects of different manipulations are assessed under common testing conditions (cf. Davis & Wagner, 1968; Gottlieb, 2008). Such common tests allow more general conclusions to be made about the impact of extinction contingencies on learning independent of the current conditions for performance. These common testing approaches have revealed that differences that appear in performance curves during different phases of conditioning and extinction do not always reflect underlying differences in learning (e.g., Drew, Yang, Ohyama, & Balsam, 2004). Further complicating the issue of testing conditions is the demonstration that even testing under common contingencies may not accurately capture differences in learning if those testing conditions differ in their similarity to the conditions of acquisition or extinction (e.g., K. M. Lattal, 1999; Stout & Miller, 2007; Wilkinson, Lee, & Bevins, 2009). Assessing the long-term impact of extinction is further complicated because many Pavlovian and operant experiments have demonstrated that repeated extinction tests often reveal savings in the speed with which the response declines over these tests (e.g., Anger & Anger, 1976; Stafford & Lattal, 2009).

Behavioral effects during and after extinction

The classic observation from extinction experiments is that over the course of extinction, learned behavior decreases from high levels to low levels. Although this finding is ubiquitous, there are multiple caveats to this observation. First, arranging extinction experimentally does not always result in an immediate loss of responding and may, in fact, result in a potentiated response early in extinction (the “extinction burst”). Second, the changes that occur during extinction will be strongly influenced by the contingencies that are arranged prior to extinction. Third, complications in comparing extinction when pre-extinction responding is at different points on the behavioral scale, as noted in the preceding subsection, prevent strong conclusions from being drawn about the shape of the extinction learning curve. Finally, although responding in the presence of the stimulus undergoing extinction may decrease, other responses may emerge during the course of extinction. We consider some of these caveats next.

Resistance to extinction: I. Response potentiation and conditioned reinforcement

Following some pre-extinction conditions (especially continuous reinforcement of an operant response), but perhaps not under all such conditions (e.g., Lerman & Iwata, 1995), responding at the onset of extinction may increase transiently relative to that observed in the pre-extinction condition. Furthermore, in Pavlovian preparations, brief extinction trials (e.g., a 1-s as opposed to a 30-s CS duration) may potentiate, rather than reduce, conditioned responding (Rohrbaugh, Riccio, & Arthur, 1972). Although both of these response potentiating effects occur, some analyses have suggested these effects to be equivocal in both Pavlovian and operant preparations (Lerman & Iwata, 1995; Rohrbaugh, et al. 1972). In addition, Skinner (1938) showed that punishing the first few responses of extinction (and thereby preventing a response burst at the outset of extinction) did not alter the ultimate time course of that extinction.

In both Pavlovian and operant preparations, the speed and extent of response elimination in extinction depends on the similarity between stimulus conditions in effect before extinction and those in effect during extinction. One such stimulus condition is the rate of reinforcement of an operant response. The removal of frequent or relatively infrequent reinforcement, for example, may itself serve as discriminative stimulus controlling response probability. Thus, differences in extinction as a function of differences in reinforcement rate could be at least in part a discriminative in addition to a nonreinforcement effect.

In another demonstration of the role of discriminative stimuli associated with reinforcement during extinction, Kelleher (1961) showed that continuing to present an empty food hopper during extinction resulted in more persistent responding during extinction than occurred when the empty food hopper was not presented in extinction. This presumably occurred because the auditory and visual cues associated with the food hopper signaled the delivery of food, resulting in those cues acquiring conditioned reinforcing value. These and other studies demonstrate that the presence of conditioned reinforcers may prolong the maintenance of responding during extinction (see Fantino & Romanowich, 2007; Robinson & Berridge, 2008; Williams, 1994).

Resistance to extinction: II. Response maintenance as a function of acquisition history

Resistance to extinction refers to the persistence of the responding established in the pre-extinction condition once extinction contingencies are in effect. Such persistence is measured in terms of the rate at which responding is reduced and the extent to which it is eliminated relative to the pre-extinction condition. Resistance to extinction can be affected by many variables, a review of which is beyond the scope of this review (see Nevin & Grace, 2000; Podlesnik & Shahan, 2010). Some of these variables in the pre-extinction condition may immunize, at least temporarily, responses to the attenuating effects of extinction in both Pavlovian and operant preparations (e.g., Bouton & Swartzentruber, 1991; Konorski, 1967; Mazur, 1994; Rescorla, 2003; Skinner, 1938; see Kimble, 1961 for a history of early such comparisons).

One of the best contemporary examples of an integration between Pavlovian and operant accounts of extinction is Nevin’s analysis of behavioral momentum and response strength as they relate to resistance to extinction. Operant responding first is stabilized under different reinforcement conditions in the presence of different discriminative stimuli (i.e., a multiple schedule). Following this, the resistance of responding in the presence of each of the stimuli is examined when disrupting events like pre-session access to the reinforcer or response-independent presentations of the reinforcer between multiple schedule components are scheduled. Most germane to the present discussion, responding in the presence of each stimulus is extinguished and differential resistance of responding in the two conditions, along with the results of the other tests, is taken to index differential strength of responding maintained by the contingencies in either stimulus condition. Using an identical preparation, Cohen, Riley, and Weigle (1993) showed, with both rats and pigeons, that there was no direct relation between rate of reinforcement on either VI, fixed-interval, variable-ratio or fixed-ratio schedules and subsequent resistance to extinction when the different reinforcement rates were varied across successive conditions. When, however, the reinforcement rates were programmed in different alternating components of a multiple schedule, orderly relations between rate of reinforcement and relative resistance to extinction were found.

Cohen et al.’s findings are significant because they lend to support to Nevin’s suggestion that resistance to change, as measured by resistance to extinction or in other ways, results from Pavlovian contingencies generated when operant responding is differentially reinforced in the different components of a multiple schedule because of the differential pairing of the discriminative stimuli with different conditions of reinforcement. Nevin, Tota, Torquato, and Shull (1990) reinforced key pecking of pigeons on a multiple VI VI schedule of food reinforcement. Adding response-independent food presentations in one component lowered response rates in that component relative to those maintained in the other component with less frequent reinforcement. Resistance to extinction, however, was greater in the component with the more frequent food delivery, even though response rates were lower in that component. Nevin et al. suggested that adding the response-independent food deliveries enhanced the Pavlovian stimulus-food contingency, with resulting greater resistance to change (see also Shull, Gaynor, & Grimes, 2002). Empirically, such research on behavioral momentum provides a test of how different contingencies differentially protect responding from extinction (see, e.g., Konorski, 1967).

Response decrement during extinction: Are extinction curves negatively accelerated?

Data from a host of experiments have shown how the pre-extinction condition influences behavior change during extinction. Despite such differences in rates of extinction as a function of manipulations during the pre-extinction condition, the form of the extinction curve appears to be qualitatively similar after these manipulations. Indeed, visualizing the thousands of extinction curves that are plotted in both the Pavlovian and operant literatures, it is striking to see how performance during extinction follows the familiar negatively accelerated pattern that is so often observed during acquisition, with a large amount of change early and smaller amounts of change as extinction progresses and responding reaches asymptote. Nevin’s analysis of behavioral momentum and response strength described above suggests that, despite the qualitative similarities in extinction curves based on operant responses, quantitative analyses of responding over the course of extinction indicates differential effects of pre-extinction variables. Relatedly, contemporary investigators in the Pavlovian tradition have noted that what often are labeled “learning curves” are more usefully labeled “performance curves” (e.g., Rescorla, 2001a). Some measure of behavior is the dependent variable and the goal is to make inferences about learning from the observed behavioral indices.

As noted previously, comparing changes in behavior from different parts of the response scale is difficult. Consider comparisons of initial acquisition, in which behavior starts at or near zero and ends at a higher level, with extinction, in which behavior starts at that high level and ends at or near zero. Based on absolute changes, one might be tempted to conclude that similarly large amounts of learning occur during the beginning of acquisition or extinction (when changes in behavior are large) compared to the end of acquisition or extinction (when changes in behavior are small). The difficulty in making inferences about the underlying learning process is that assumptions must be made about how a similar increase or decrease in units on a behavioral scale (e.g., a gain or loss of 10 responses per minute on a given trial) maps on to the underlying learning. Scaling issues make it impossible to know with any precision what the relation is between the observed performance and hypothetical learning functions.

In a series of experiments, Rescorla (e.g., 2001a, 2002a, 2002b) developed a novel test procedure for evaluating the potential differences in learning that occur during early and late parts of acquisition and extinction. By testing target stimuli that have different histories of acquisition and extinction in compound with stimuli that have received common treatment (and thus evoke common levels of responding), one can infer that differential effects on compound summation reflect differences in learning to the target stimuli. This approach has revealed that the negatively accelerated performance functions associated with acquisition and extinction do, in fact, reflect differences in underlying learning, with larger changes early compared to later in the behavioral treatment. This difference between the effects of early and late trials of extinction that has been so influential in Pavlovian analyses also is evident in operant experiments. For example, Hearst, Besley, and Farthing (1970) showed that inhibitory stimulus generalization gradients were steeper earlier than they were later during the course of extinction (c.f., Rescorla, 2002a). Together, these findings suggest that the impact of an extinction trial will differ depending on when in the course of extinction it occurs.

Response-generative effects of extinction

Much of the focus of research on extinction has been to examine the effects of different extinction contingencies on the stimulus or response that is the target of those contingencies. The dependent variable is usually changes in the response that was established during acquisition. Extinction, however, not only reduces or eliminates the targeted response, but it also generates or occasions increased variability in behavior. This variability can include topographical variations on the previously reinforced response (Antonitis, 1951) or the appearance of quite topographically distinct responses (e.g., Tinsley, et al 2002).

Increases in response variability in the presence of the extinction stimuli are commonly reported. Replicating an earlier experiment by Antonitis (1951) using rats, Eckerman and Lanson (1969) reinforced each response (a fixed-ratio 1 schedule) of pigeons at any point along a 10-in long strip composed of 20 separate response keys. During the FR 1 schedule, responses tended to recur along a narrow band of the strip, but during extinction responses occurred across a wider band of the strip and, for two of three pigeons, were more evenly distributed across this wider band.

In addition to variability in responding in the presence of target stimuli, extinction can induce new responding to other stimuli that are present during extinction. The classic example of this comes from Azrin, Hutchinson, & Hake (1966), who found that the onset of periods of extinction of key-peck responses of pigeons was accompanied by an increased frequency of attack on a restrained target pigeon by the pigeon whose responding was undergoing extinction. Such induced behavior is observed not only when an extinction procedure is formally implemented, but any time there is a signaled period of extinction. The form of the behavior conforms to the target object – for example, drinking if a lick tube is present, running if a running wheel is present, or pica if a material appropriate for pica is present. Such extinction-induced responding is not limited to the complete elimination of reinforcement. Within a schedule of reinforcement, for example, discriminated periods of nonreinforcement also may induce other-than-operant responses (e.g., Falk, 1966).

These variations in previously reinforced response topography and extinction-induced responding may be conceptualized as instances of resurgence (see da Silva, Maxwell, & Lattal, 2008, and Winterbauer & Bouton, 2010, for recent examples). Originally labeled regression by Keller & Schoenfeld (1950; see also Carey, 1951), the contemporary procedure for demonstrating resurgence is to first train and then extinguish a response. Either concurrent with or following the latter extinction, an alternative response is reinforced. Extinction of the second response then results in a transient reappearance of the still-nonreinforced first response, which defines resurgence. This behavioral process may operate generally when a response is extinguished: through a process of resurgence, previously learned responses may recur when the target response is no longer reinforced

Resurgence occurs in the same stimulus conditions that were in effect during the pre-extinction condition. Extinction also can induce changes in behavior in other stimulus conditions than those in which extinction is occurring. Behavioral contrast is an example. Following training of responding in the presence of two alternating stimulus conditions (a multiple schedule), the schedule in one of the conditions is replaced with extinction. Responding is not only reduced in that component, but it increases in the other component – behavioral contrast (Reynolds, 1961). Despite its label, reductions in response rate without concurrent reductions in reinforcement rate do not yield contrast (Halliday & Boakes, 1971). This has led to suggestions by some that Pavlovian stimulus-reinforcer relations are important for behavioral contrast (e.g., Gamzu & Schwartz, 1973; Schwartz, 1977; McSweeny, Ettinger, & Norman, 1981; but see Williams, 1983; Williams & McDevitt, 2001). However labeled, contrast has generality across different reinforcement conditions maintaining responding in the unchanged component, response classes, and species (see Williams (1983) for a review), although much of the research on behavioral contrast has, in fact, been conducted using pigeons.

Long-term effects of extinction

A pervasive finding regarding the learning that occurs during extinction is that extinguished behavior returns under many conditions. The earliest documentations of this come from Pavlov (1927) and Skinner (e.g., 1938). Pavlov found that extinguished conditioned responses return with time (spontaneous recovery effects; see also Mazur, 1994; Rescorla, 2004), or with post-extinction presentations of the US or reinforcer (reinstatement effects; see also Franks & Lattal, 1976; Rescorla & Heth, 1975). Pavlov also reported an early case of a situation in which the extinguished conditioned response returned when testing moved from the extinction context (the laboratory) to a different context (a classroom; renewal effects, see also Bouton & Bolles, 1979; Nakajima, Tanaka, Urushihara, & Imada, 2000). Both recent findings and the early work of Pavlov and Skinner demonstrate that the loss of the behavior that occurs during extinction is subject to constraints. As the conditions after extinction become more similar to the conditions prior to extinction, responding often returns.

This effect of pre-extinction and post-extinction similarity on responding is nicely illustrated in operant studies by Uhl and Garcia (1969) and Hearst, et al. (1970), who used what Hearst et al. labeled a “resistance to reinforcement” test of the efficacy of extinction in eliminating responding. Following the attenuation of responding to low levels, they reinstated the conditions in effect prior to extinction and tracked the course of the return of responding. Uhl and Garcia eliminated lever press responding previously maintained by a variable-interval (VI) schedule either by removing the reinforcer (extinction) or by a response-omission procedure whereby each lever press postponed reinforcement delivery. After responding was reduced to near-zero, either the VI schedule or, for some subjects, a variable-time schedule that delivered response-independent food was reinstated. The previously eliminated responding returned at approximately the same rate and level regardless of whether it had been extinguished or reduced by the response-omission procedure. Although the procedure is not widely used to assess extinction effects, this resistance to reinforcement procedure could prove useful in testing the long-term effects of extinction following other pre-extinction conditions.

Studies by Delamater (1997) demonstrate that reinstatement is due to the contribution of general processes on responding, as well as to re-establishment of responding associated with specific outcomes. In Delamater’s experiments, two CSs were associated with two USs (e.g., tone-sucrose and noise-pellets), then both were extinguished. Following extinction, presentation of sucrose reinstated some level of responding during the noise (demonstrating some outcome-independent reinstatement). But, importantly, sucrose caused more responding during the tone compared to the noise (demonstrating outcome-selectivity in reinstatement). Experiments like these reveal that the content of the CS-US association is preserved during extinction and that reinstatement occurs through contributions of general outcome-independent processes, but also to reintroduction of the specific outcome used in the acquisition phase.

Theoretical analyses of extinction

Two major theoretical research programs (Lakatos, 1970) or traditions (Laudan, 1977) in the study of learned behavior are associationist (e.g., Mackintosh, 1983) and behavior analytic or operant (e.g., Skinner, 1938). For both, a central question is: Why does extinction occur?

In the operant tradition, the causes of extinction most often are to be understood through a thoroughgoing experimental analysis of the conditions under which extinction occurs. Skinner (1938) originally conceived of extinction as metaphorically draining the reflex reserve created when responding was being reinforced. He subsequently abandoned this mechanistic view of extinction, with what he considered to be its unnecessary theoretical constructs, in favor of a functional approach to extinction: it is to be understood by identifying the variables of which it is a function. Ferster and Skinner (1957), for example, maintained responding under different schedules of reinforcement and then observed the effects of discontinuing the reinforcer on the persistence of response rates and patterns.

Others have examined in detail the role of reinforcement parameters, behavioral histories, and stimulus variables as they contribute to the occurrence of operant extinction (for a review see Lattal, St Peter, & Escobar, in press). As already noted, Nevin’s analysis involving extinction, described in the Resistance to extinction: II. Response maintenance as a function of acquisition history section is a major contribution to the functional understanding of extinction. The importance of context in accounting for operant extinction is illustrated by Kelleher’s (1961) experiment, described in the Resistance to extinction: I. Response potentiation and conditioned reinforcement section above, showing how similarity between pre-extinction and extinction conditions affects the course of extinction, a point that also is illustrated by the conditions that promote operant reinstatement (Franks & Lattal, 1976). Indeed, much of what is known about the general principles of operant extinction come from a functional approach in the Skinnerian tradition. This approach is not without its critics, however, even within behavior analysis (e.g., Staddon, 1993). Some theories of operant extinction include such hypothetical constructs as competing responses (e.g., Adelman & Maatsch, 1955) and inhibition (Hearst et al., 1970). Nonetheless, this generally functional approach has advanced both the understanding of extinction and the use of extinction in a host of applications.

Associationist theories of extinction are grounded in empirical findings from both Pavlovian and operant extinction, with much of the recent theorizing coming from the Pavlovian analyses (reviewed in Rescorla, 2001b). One of the key assumptions of modern views of Pavlovian conditioning is that both initial conditioning and extinction occur as a function of the error between the reinforcer that is predicted based on the available cues and the reinforcer that is obtained. This notion of prediction error is classically illustrated in the Rescorla-Wagner (1972) model of conditioning, although models before and after have capitalized on the predictive error idea in different ways (Bush & Mosteller, 1955; Mackintosh, 1975; Pearce, 1987). By this view, extinction occurs because the reinforcer that becomes anticipated during acquisition is omitted, which creates a negative prediction error, resulting in the loss of behavior. This approach has led to postulation of novel mechanisms involved in operant reinforcement and extinction (e.g., K.M. Lattal & Nakajima, 1998; Williams, 1984) and testing the implications of this and other theoretical approaches to extinction has been fruitful for the development and refinement of general quantitative theories of extinction (see Killeen, Sanabria, & Dolgov, 2009).

The importance of negative prediction error in driving extinction is central to many theories, but the prediction error concept itself does not speak to the inhibitory association that is assumed by some theorists to underlie extinction. That extinction is mediated by some inhibitory mechanism is common to many theories, including some theories of extinction promulgated by those grounded in the behavior-analytic tradition (e.g., Hearst et al., 1970). Inhibition, however, generally is more central to and conceptually developed in associationist theories. Different versions of these theories place differing emphasis on inhibition of the conditional stimulus itself (e.g., Robbins, 1990; Schmajuk & Larrauri, 2006), the unconditional stimulus itself (e.g., Rescorla & Heth, 1975; Stollhoff & Eisenhardt, 2009), and different combinations of conditional and unconditional stimuli and responses (e.g., Rescorla, 1993; see Delamater, 2004 for a review).

In addition to appealing to inhibition in relations between stimuli, responses, and outcomes, some theories suggest that inhibitory learning about the context in which extinction occurs is important for mediating changes in behavior during extinction. There is some evidence that extinction of a Pavlovian CS results in the context of extinction gaining inhibitory value (e.g., Polack, Laborda, & Miller, 2011), though other evidence suggests that contextual inhibition is not necessary for extinction (e.g., Bouton & Swartzentruber, 1986). Bouton (2004) suggested that context mediates or controls responding by serving as a retrieval cue for the learning that occurs during extinction. Bouton’s theory broadens the traditional definition of context to include not just external physical characteristics (such as visual and tactile cues), but also long-term and short-term temporal contexts (e.g., long temporal intervals, like those between the end of extinction and testing for spontaneous recovery, and short temporal intervals, like those between extinction trials; Bouton & Garcia-Gutierrez, 2006), and internal contexts (such as drug states; e.g., K.M. Lattal, 2007). This theory, which was initially developed to account for a specific data set related to Pavlovian conditioning and extinction, has been useful for making novel predictions about the nature of extinction of operant behavior, including extinction in applied settings (cf. Crombag, Bossert, Koya, & Shaham, 2008).

Operant accounts of extinction similarly emphasize the importance of discriminative stimulus control of responding during extinction as a function of discriminative stimulus-reinforcement relations in pre-extinction conditions (e.g., Skinner, 1950; Nevin et al., 1987). Incorporating these different aspects of context into one theoretical approach has lead to the development of general theories about operant and Pavlovian extinction (e.g., Bouton & Swartzentruber, 1991). These theories emphasize the importance of conceptualizing extinguishing or extinguished responding in different ways, with the goal of identifying similar principles to account for the occurrence of behavior during pre-extinction, extinction, and post-extinction testing.

Applications of extinction

Outside of the laboratory, the use of extinction as a tool for eliminating undesirable behavior has been extremely successful. At the clinical level, extinction combines Pavlovian and operant approaches that are focused on weakening cue-evoked reactions. There are countless examples of successful applications of extinction (e.g., Rothbaum & Schwartz, 2002), though there are also examples of failures (see Conklin & Tiffany, 2002). These failures may be due to various factors, but certainly, the general demonstration that the behavior changes induced by extinction may not persist across time and contexts illustrates some of the challenges with using extinction as a clinical behavioral intervention.

Indeed, the progress that occurs during extinction-based therapies may reverse as time passes or when certain cues are encountered (e.g., physical contexts, people, drugs) that are associated with the behavior that was extinguished. Clinical applications of extinction have recognized the importance of the contextually dependent nature of extinction by attempting to arrange treatment in a way that promotes the generalization of extinction outside of one particular context (see, e.g., Stokes & Baer, 1977). Strategies include extinction in multiple contexts (e.g., Gunther, Denniston, & Miller, 1998; but see Bouton, Garcia-Gutierrez, Zilski, & Moody, 2006) and associating particular reminder cues with extinction that can then be used outside of the extinction context to recall the extinction memory (see Brooks, 2005).

Extinction is not only one of the oldest forms of treatment for eliminating or reducing behavior (see Fuller 1949, for an early demonstration of operant extinction with a human), but it is the basis for other useful treatment techniques. It is, for example, incorporated into both differential-reinforcement-of-other behavior (DRO) and differential-reinforcement-of-alternative-behavior (DRA). Both techniques have become more in vogue for treating problem behavior because of ethical concerns associated with simply eliminating reinforcers for negative behavior without concurrently attempting to reinforce some more positive behavior. In these techniques, until the first reinforcer is encountered for either not emitting the target response or emitting the alternative response, the procedure is conventional operant extinction. Thus, in implementing treatment programs involving these procedures, consideration must be given to both stimulus and outcome variables that affect the rate and ultimate level to which responding might be reduced, as well as those related to the resistance of extinguished responding to recovery once treatment is terminated (as discussed above).

Furthermore, the potential generative effects of any extinction-based treatment must be considered. Depending on the alternative behavior generated either in the presence of the stimuli associated with the extinction contingency or in the presence of other, related environments, generative effects of extinction can be a blessing or a curse. As the targeted behavior is eliminated, other more positive responses may emerge that then can be reinforced. Alternatively, the behavior that emerges may be as equally unacceptable as, or worse for the client’s well-being, than the targeted response (see K.A. Lattal & St. Peter Pipkin, 2009).

Conclusions

Extinction is a procedure and a process that is significant for the empirical and theoretical study of learning and for its practical applications. Understanding how behavior changes during extinction has been key to understanding many phenomena of Pavlovian and operant acquisition and maintenance. Describing the learning processes that underlie extinction has been central for development of both general descriptive accounts of behavior and for theoretical approaches that speculate about the mechanisms that operate during learning. Many of the methodological issues described in this review come up repeatedly in the analysis of extinction, independently of behavioral preparation or theoretical perspective. In both domains, there also has been a great appreciation that many factors must be considered in analyzing the changes in behavior that occur when environmental contingencies change. As research on Pavlovian and operant extinction continues, additional points of contact will emerge that will suggest further unifying principles and clinical applications.

Highlights.

• This article reviews research and theory on Pavlovian and operant extinction.

• This article focuses on common procedural and interpretational challenges in Pavlovian and operant extinction.

• This article addresses applications of Pavlovian and operant research on extinction.