Resurgence of Response Sequences during Extinction in Rats Shows a Primacy Effect

Abstract

Rats were trained to emit a series of three-response sequences to a criterion (i.e., more than 80% of all emitted sequences correct over five successive sessions). Each rat was trained on a series of different, three-response sequences. After the final three-response sequence was acquired, two extinction tests were administered, and the three-response sequences that re-emerged during these extinction tests were noted. Resurgence effects during extinction were observed; that is, the previously trained sequences were emitted. These resurgence effects followed an orderly pattern, which involved a primacy effect. The rats initially emitted the immediately previously trained response, but then started to emit the response sequence they first were trained to emit. Thus, resurgence behavior during extinction can be an orderly function of previous training history. These results replicate those previously obtained with human subjects.

An issue of contemporary importance in behavior analysis is how “history variables” interact with the currently prevailing contingencies. History variables are of interest because behavior from a subject's prior conditioning history can resurface during an experiment (e.g., Tatham & Wanchisen, 1998; Wanchisen, 1990). Indeed, much of an organism's current behavior may have its roots in the specific details of its learning history. One such history effect is termed “resurgence”, defined as the reappearance of behavior patterns observed earlier in a subject's learning history, but which are not observed in the subject's more recent history. There are a number of different terms that have been applied to describe the re-emergence of historically trained behavior into the organism's current behavior, and these terms usually are defined by reference to the procedure that produces the re-emergence. For example, “resurgence” typically refers to previously established behavior that re-emerges during extinction. In contrast, “reinstatement” often refers to the re-emergence of previously trained behavior as a result of the delivery of response-independent reinforcement. It is presently unclear if different mechanisms are responsible for these history effects.

Previously documented examples of resurgence phenomena were concerned with deviations from practiced routines, errors during the performance of skilled behavior, and complex relational learning in humans. Mechner, Hyten, Field, and Madden (1997; see also Mechner, 1992) showed resurgence of previously learned response sequences in humans. Additionally, resurgence of derived relations has been demonstrated in research on equivalence classes (Wilson & Hayes, 1996). A number of conditions responsible for resurgence have been identified. Epstein (1985) cites punishment, satiation, and increased response requirements as conditions that may induce resurgence. Epstein (1983) also showed that resurgence of previously reinforced responding occurred when currently reinforced responding was extinguished. Mechner (1992) discussed resurgence in the context of the reappearance of previously punished behavior, and he suggested that resurgence provides part of the explanation for the increase in variability normally seen in extinction, or under other conditions of stress. In addition, it has been found that the contingencies that prevail during acquisition, or the nature and amount of reinforcement used, can affect the amount and nature of resurgence during extinction (Dixon & Hayes, 1998; Pittenger, Pavlik, Flora, & Kontos, 1988).

Typically, the experiments cited above with human subjects have studied resurgence using complex response patterns, that is, responses that involve more than a single discrete action. History effects have been examined with nonhuman subjects, and the role of numerous factors in the development of the re-emergence phenomenon has been addressed. For example, Doughty, Reed, and Lattal (2004) and Franks and Lattal (1976) noted reinstatement of previous responding under conditions of response-independent reinforcement. In these studies, behavior (simple lever pressing or key pecking) was established in subjects (rats and pigeons) through response-dependent delivery of food. The responses then were extinguished, and following elimination of the behavior response-independent food was delivered. This manipulation reinstated responding in the subjects. There are few studies, however, that have employed extended sequences of behavior in a manner that would allow examination of the re-emergence of complex response patterns, such as are used in experiments employing humans. There have been experiments showing that rats will learn successive response sequences (e.g., Fetterman & Stubbs, 1982; Reed, Schachtman, & Hall, 1991), but these studies did not test for the subsequent resurgence of the response sequences. Moreover, there are very few studies in nonhumans that have examined the resurgence of responding (that is, the re-emergence of previously trained responding during extinction), as opposed to the reinstatement effects above (in which previously observed behavior, which has been extinguished, re-emerges usually due to the delivery of response-independent reinforcement). The current experiment extends the human operant procedures used previously to nonhuman subjects, in a resurgence paradigm, in order to better integrate the nonhuman and human literature.

Although the primary aim of the current study is to integrate the literature on resurgence effects studied in humans and nonhumans with respect to the use of complex or simple responses, the current work also allows assessment of a secondary issue that has arisen in some early reports of such work and previously published work on response-sequence learning in nonhumans. In a related piece of work, Reid (1994) successively trained rats to emit three-response sequences. It was noted that retraining a new sequence proceeded faster if the last response in the three-response sequence was changed compared to when the first response in the sequence was changed. Reid suggested that this finding implied greater response strength accrued to the last response in the sequence compared to the first. If this explanation is accepted, then it suggests a recency effect is obtained. This result is consistent with that of an unpublished study from our laboratory (Reed & Morgan, 2006). When the behavioral patterns induced by schedules of reinforcement have been used in resurgence studies, the behavioral patterns emitted during testing recapitulated in reverse order their sequence of acquisition (i.e., behavioral reversion).

In the Reed and Morgan (2006) study, rats initially were trained to respond on a multiple random-ratio (RR) random-interval (RI) schedule, which established large response-rate differences between the two components (i.e., higher rates to the RR compared to the RI schedule). Following this training, rats were exposed to a multiple fixed-interval (FI) FI schedule, with equal FI values in each component. Rats responded on this schedule until response-rate differences between the components disappeared. The rats then were placed on a multiple extinction (EXT) EXT schedule. During extinction, response rates were higher in the component previously associated with the RR component of the multiple RR RI schedule. These data suggest that the last trained response sequence re-emerges first, similar to that effect noted by Reid (1994). In contrast, recent work with humans (e.g., Mechner & Jones, 2001) has indicated that resurgence of previously learned response sequences shows both primacy and recency effects. That is, the response sequences that were learned first in the sequence (primacy) and last in the sequence (recency) showed greatest levels of resurgence in the test phase. There are many differences between these two sets of experiments (i.e., Mechner & Jones, 2001; Reed & Morgan, 2006; Reid, 1994). The current study eliminates some of these procedural differences in order to allow a clearer comparison of the resurgence effects across species.

The particular operant classes in the present study consisted of an extended sequence of responses to levers. Even though the operant as a whole is composed of more than a single response, the entire sequence of actions can be regarded, with empirical justification, as a single operant unit (see Mechner et al., 1997; Reed et al., 1991; Schwartz, 1982). There are a number of sound reasons for adopting this approach. As noted above, both humans (Mechner et al., 1997) and nonhumans (Fetterman & Stubbs, 1982; Reed et al., 1991; Reid, 1994) readily acquire such sequences, which serve as integrated operants (Reed et al., 1991; Schwartz, 1982). This allows for the integration of work from both human and nonhuman studies using a similar methodological base.

In the current experiment, rats were trained to perform a series of response sequences. The response sequences consisted of a number of individual responses (left and right lever presses in an operant conditioning chamber). Once the sequences were performed to a set criterion, the rats were taught a new sequence. After the successive acquisition of a number a sequences (three), the rats were transferred to a test condition, comprising a short extinction session. The order in which the sequences were performed by the rats during this test session was recorded. Should the previously obtained results from human subjects involving response sequences be replicated, the rats should emit greater numbers of the first and last sequences trained during the test sessions.

Method

Subjects

Six male, Lister hooded rats were employed. The rats previously were trained to lever press and retrieve food from the food magazine. They were approximately 8-9 months old at the start of the experiment, and had a free-feeding body weight range of 405-420 g. The rats were housed in groups of four and were fed to maintain their 85% free-feeding body weights. Constant access to water in the home cage was provided.

Apparatus

Four identical operant conditioning chambers were used. Each chamber was located in a sound- and light-attenuating box equipped with a ventilation fan that provided a background masking noise of 65 dB(A). Each chamber measured 235 mm wide × 235 mm long × 205 mm high. On one wall of the chamber were two identical stainless-steel response levers located 30 mm to each side of a centrally located food magazine and 30 mm from the floor. The food magazine was covered by a hinged, clear Plexiglas flap, behind which reinforcement (one 45-mg Noyes food pellet) could be delivered. The magazine was not illuminated at any point during the experiment. A jewel light was positioned 30 mm above each lever.

Procedure

The subjects received two sessions of magazine training on a variable-time 60-s schedule, with both levers retracted from the chamber. They then received two 20-min sessions of lever-press training on a continuous reinforcement (CRF) schedule. In one session the left lever, and in the other session the right lever, was extended (the alternate lever was withdrawn from the chamber during these sessions). All rats were pressing reliably during each session with each lever.

Following this pretraining, the subjects were trained to emit the first response sequence. Initially, one lever was inserted into the chamber, and the light above that lever was illuminated. This lever was to form the final lever of the sequence for that particular subject. The rats were given three sessions of CRF training with this lever, each lasting until they had made 75 responses. After each response was made, the light above the lever was extinguished, and food was delivered. There was a 5-s intertrial interval (ITI), during which responses were not reinforced. The light was then illuminated again, and a response would lead to the delivery of food.

Following this training, the rats were trained to emit a two-response sequence. This sequence comprised the middle and terminal responses of the target sequence to be trained. During this training, both levers were inserted into the chamber, and the light above each lever was illuminated. After the subject made two responses, the lights were extinguished. There were no explicit stimulus changes between the responses. If the rat had emitted the correct s equence, then food was delivered, a 5-s ITI ensued, and the lights would be reilluminated (see Table 1 for summary of the sequences employed). If the rat had emitted an incorrect sequence, the lights would be extinguished, no food was delivered, and a 5-s ITI ensued. Each session lasted until the rat had emitted 150 responses. This phase of training lasted until 80% of all sequences (i.e., 60/75) were correct for three successive sessions. The range of sessions required to complete this training for the rats was 25 to 42.

Table 1. Response sequences trained in each rat along with the number of sessions to criterion (shown in parentheses). L and R refer to a left and right lever press.

Phase	Rat A	Rat B	Rat C	Rat D	Rat E	Rat F
1	LLR (57)	LRL (63)	RLL (42)	RRL (74)	RLR (57)	LRR (81)
2	LRL (69)	RLL (90)	LLR (45)	RLR (69)	LRR (86)	RRL (69)
3	RLL (89)	LLR (76)	LRL (63)	LRR (57)	RRL (70)	RLR (99)

Once the rat had acquired the two-response sequence, it was trained on the target three-response sequence. Sessions proceeded as described above, with the exception that sessions lasted until 225 responses (i.e., 75 three-response sequences) had been made. Subjects were trained until over 80% of all sequences emitted were correct (i.e., 60/75 sequences correct) for 20 successive sessions.

Once the rats had satisfied this criterion they were shifted directly onto the second of the three-response sequences that they were to emit (i.e., they did not move through one-response and two-response sequence training as in Phase 1). Training continued until they emitted 80% of all sequences correctly for 20 sessions. The same stimulus conditions were employed in this second phase of training (i.e., the second phase was not differentially signaled by any cue). Finally the subjects were trained on the third and final three-response sequence (Phase 3), until they met the same criteria. The exact nature of the three response sequences trained for all rats, along with the number of sessions of training required to reach this criterion are shown in Table 1. The sequences were chosen to represent all the possible combinations of three-response sequences (excluding sequences comprising three responses on the same lever). Each rat received three different sequences, randomly determined, but with the provision that no rat received the same three sequences as another rat.

Once the rats had completed all three phases of training, they were given an extinction test, during which the levers were presented with the lights illuminated, but food reinforcement was discontinued. The rats were allowed to make three responses, and the lights extinguished. A 5-s ITI ensued, but no food was delivered. The sessions comprised 20 such trials. There were two consecutive extinction sessions; one conducted on each of two consecutive days. Only two short extinction sessions were conducted in order to prevent performance from becoming too variable making the data potentially uninterpretable.

Results

Figure 1 shows for each subject the number of three-response sequences emitted during each extinction session that conformed to one of the previously trained sequences. During the first test session, 65% to 80% of all the sequences emitted during that test session were of the types that had been trained previously. None of the five untrained three-response sequences (which differed for each rat) was emitted more than three times for any subject during this first test session. It should be noted that chance levels of emission for one of the eight possible sequences in a 20-trial test session would be 2.5 emissions. It is worth noting that as the extinction sessions progressed (i.e., from session 1 to session 2) greater numbers of nontrained responses came to be emitted. During the final training phase, only a mean of 7% of all sequences emitted were nontrained. This group mean increased to 26% during the first test phase, and then to 49% during the second extinction test.

Fig 1. Number of previously trained response sequences emitted by each rat on each extinction test session.

Of the previously trained three-response sequences, all rats emitted the final (Phase 3) trained three-response sequence (Sequence 3) more than the other two previously trained sequences, and emission of the previously trained final sequence was well above chance levels. Three rats (Rats A, D, and E) next most often emitted the first three-response sequence trained (Sequence 1). Levels of emission of this sequence were above chance levels (but levels of emission of the sequence trained second in the training phase (Sequence 2) were below chance levels of emission). For 2 rats (Rats B and F), the Sequence 2 was the next most often emitted sequence during the first test session, and this sequence was emitted above chance levels (whereas Sequence 1 was not emitted above chance levels for these latter two rats).

Analysis of these data by a repeated-measures analysis of variance (ANOVA) revealed a statistically significant difference between the levels of emission of the three sequences, F(2,10)  =  27.10, p < 0.001. Protected t-tests, conducted on each pairwise comparison of sequences, revealed that Sequence 3 was emitted more than both Sequence 2, t(5)  =  9.80, p < 0.001, and Sequence 1, t(5)  =  4.87, p < 0.005. However, there was no statistically significant difference between Sequence 2 and Sequence 1, t(5)  =  1.30 , p > 0.20.

The data from the second extinction test session show a different pattern of results. During this session, the number of previously trained sequences ranged from 40% to 60% of all sequences emitted by the rats. None of the untrained sequences was emitted more than four times during a session by any rat, and, in almost all cases, the level of emission of these untrained sequences was close to chance levels. During the second test session, the subjects emitted Sequence 2 more often than the other two sequences. In all cases, emission of the second trained sequence was above chance levels.

Analysis of these data from the second test session by a repeated-measures ANOVA revealed a statistically significant difference between the levels of emission of the three sequences, F(2,10)  =  17.71, p < 0.001. Protected t-tests, conducted on each pairwise comparison of sequences, revealed that Sequence 2 was emitted more than both Sequence 3, t(5)  =  3.95, p < 0.001, and Sequence 1, t(5)  =  5.86, p < 0.005. However, there was no statistically significant difference between Sequence 3 and Sequence 1, t(5)  =  1.09 , p > 0.30.

Comparing the change of level of sequence emission from the end of Phase 3 training to the first extinction test session, the level of emission of Sequence 3 decreased from a mean of 85% of all sequences emitted to a mean of 49%, t(5)  =  6.85, p < 0.001; the mean level of Sequence 2 emission rose slightly, but not statistically significantly, from 4% to 6%, t(5)  =  1.75, p > 0.10; and the mean level of Sequence 1 emission rose sharply from 4% to 17%, t(5)  =  3.48, p < 0.05. Comparing the two test sessions, it can be seen that during the first test session there were, perhaps not surprisingly, more emissions of Sequence 3 than of the other two sequences, but that Sequence 1 also was emitted reasonably often. On the second test session, levels of emission of Sequence 3 had dropped considerably, t(5)  =  9.65, p < 0.001, and levels of emission of Sequence 2 had increased, t(5)  =  6.37, p < 0.001, whereas levels of Sequence 1 were maintained at their first test levels, t(5)  =  1.51, p > 0.10.

Figure 2 displays the number of each previously trained sequence emitted in both test sessions (session 1 top panel and session 2 bottom panel) as a percentage of all the previously trained sequences emitted during those test sessions. Inspection of these data confirm what was described above, in that they reveal strong recency (last-trained response) and a moderate primacy (first-trained sequences) effects when the data from the first test sessions are considered. However, on the second test session, most rats emitted a greater proportion of Sequence-2 responses.

Fig 2. The percentage of all previously trained sequences emitted on test session 1 (top panel) and test session 2 (bottom panel) for each rat.

Figure 3 displays the change in percentage of the previously trained sequences from the first test session to the second test session. That is, the percentage of times that a previously trained sequence was emitted in the first test session was subtracted from the percentage of times which that previously trained sequence was emitted in the second test session. For all rats, there was a large reduction in the percentage emission of the last-trained sequence between the two test sessions. When compared against zero, this change was statistically significant, t(5)  =  6.19, p < 0.005. In contrast, there was a large increase in the percentage of sequences accounted for by the second-trained sequence, t(5)  =  12.60, p < 0.001 (relative to zero), with there being little change in the percentage of sequences accounted for by the first-trained sequence across the test sessions, t(5) < 1.0 (relative to zero).

Fig 3. The percentage change in the previously trained response sequences emitted on test session 2 compared to test session 1 for each rat.

Discussion

This study was conducted to examine resurgence behavior in rats during extinction, and specifically to examine whether rats show systematic tendencies to emit response sequences during extinction that they had been serially trained to emit previously. Previous studies using human (Mechner & Jones, 2001) and rat (Reed & Morgan, 2006) subjects have produced somewhat different patterns of data, but very different experimental procedures were used in those studies. The current experiment aimed to clarify the situation by adopting for rats procedures similar to those used previously with humans. That is, a number of previously trained sequences composed of discrete responses were used as the elements of a sequence.

Primacy resurgence effects emerged from the present rat subjects when resurgence of previously trained response sequences was examined during extinction. This finding replicates that reported by Mechner and Jones (2001) who used human subjects emitting response sequences. Thus, it appears that resurgence history effects can follow an orderly pattern when organisms (human or rat) are placed in extinction. Obviously, initially the subjects continue to emit the immediately previously trained response sequence. Once the emission of this response sequence decreases, the tendency is for the earlier of the other two trained response sequences to re-emerge, followed by other response sequences trained in the middle of the serial training program. It was not the case in either the current experiment or that of Mechner and Jones that behavior during extinction became immediately chaotic; rather, behavior followed orderly patterns of resurgence. Such findings also mirror those reported by Rawson, Leitenberg, Mulick, and Lefebvre (1977). Rawson et al. studied the effectiveness of reinforcement of alternative behavior as a response-suppression technique. They noted that when reinforcement for an alternative behavior is discontinued, substantial recovery of an original response often is observed.

Additionally, the present findings replicate the primacy effects noted for a range of paradigms in nonhumans (Bolhuis & Van Kampen, 1988; Reed, Croft, & Yeomans, 1996; Williams, McCoy, & Kuczaj, 2000). Why some reports fail to note such effects is still unresolved, but the majority of findings now clearly point to these phenomena as well established in nonhumans. The processes responsible for the generation of primacy and recency effects still are not clear, but the current data argue against the possibility that more-trained responses return in extinction more readily than less-trained responses. In the current study, most of the rats received fewer training sessions for the first sequence than for the second and third sequences.

The present results also provide data that allow comparison of the two prior studies of resurgence of previously trained “complex” responses. Mechner and Jones (2001) noted primacy and recency effects when using human subjects trained to emit various types of response sequences. Reed and Morgan (2006) noted a “reversion effect” when using rats previously trained on schedules of reinforcement. Reversion refers to the re-emergence of previously observed patterns of behavior that are not currently maintained by the contingencies in reverse order to that in which they were learned. In the later study, response rates during extinction varied as a function of the order in which the rats had been exposed previously to either high-rate or low-rate producing schedules of reinforcement. The order in which these response rates resurged during extinction was the reverse order of their training. There were numerous differences between the studies conducted by Mechner and Jones and by Reed and Morgan that make direct comparison difficult. In addition to the species used (human vs. rat), and the type of complex response employed (response sequence vs. schedule), Mechner and Jones trained many sequences prior to testing, whereas Reed and Morgan only trained two schedules. The current data suggest that in the absence of other influences (i.e., in extinction), resurgence appears to follow a pattern that reflects primacy and recency effects rather than pure reversion.

There are still many further variations of procedure that could be studied: resurgence after exposure to a larger number of schedules of reinforcement; the effects of different test conditions in addition to extinction tests; and the effect of greater number of extinction tests (somewhat difficult given the cessation of responding). In addition, the current study did not use defining “start” and “end” behaviors as suggested by Mechner et al. (1997). These behaviors serve to signal that the organism is going to emit the unit (start response), and has completed the unit (end response), but they do not form part of the sequence. It also might be noted that the response sequences employed in the current study had similarities to each other, and generalization between them might have had an effect on the results. The use of sequences composed of topographically different responses (e.g., chain pulls and lever presses), or the training of spatially distinct response sequences may help with these questions. However, there are reasons to doubt that generalization between the sequences can be a sufficient explanation of the data. All the rats in the present study were trained on different response sequences, yet all produced similar patterns of data. The sequences emitted during test retained their integrity relative to their training; that is, the rats continued to emit the trained sequences with very few errors, and their emission of previously trained sequences was consistently above chance and above that of nontrained response sequences, indicating that the trained sequences appeared to retain their response integrity (see Reed et al., 1991).

It is worth comparing the current results in light of two previous and similar studies. The current results corroborate those reported by Neuringer, Kornell, and Olufs (2001) in terms of the increased variability in emission of response sequences noted as the extinction trials continued. A greater range of sequences was emitted in the second extinction trial as compared to the first, and a greater number of nontrained sequences was emitted. This pattern of result also is consistent with that noted for simple responses described by Antonitis (1951). The data also might be compared with those presented by Reid (1994), and again offer some support for the conclusions. In this latter study, it was shown that when three-response sequences were trained in sequence, responses in the last position of the sequence were more sensitive to either new learning (learning of new sequences was faster when the change occurred in the last position) or extinction (extinction of the old sequence was faster when the change was in the last position). Reid suggested that these findings implied greater response strength in the last position. If this explanation is accepted (although it could be argued on the grounds of response persistence that just the opposite was the case because the first response was more resistant to change), then the current procedure suggests that such a recency effect might be evident, but that a primacy effect should also be present. The procedures of the current study and both of the above studies are quite different, but the similarities in the data do help to place the current results in a wider context.

In summary, the present results show that resurgence effects during extinction occur in rats previously trained with complex response sequences. These resurgence effects follow an orderly pattern during extinction, involving primacy and recency effects, and replicate those seen previously in human subjects.