Context Change and Associative Learning

Abstract

The present article reviews the effects of changing the background context on performance in associative learning tasks in humans and animals. The findings are complementary and consistent over animal conditioning (Pavlovian and instrumental learning) and human predictive learning and memory paradigms. In many cases, a context change after learning can have surprisingly little disruptive influence on performance. Extinction, or retroactive interference treatments more generally, are more context-specific than the initial learning. Contexts become important if the participant is exposed to any of several treatments that involve prediction error, which may serve to increase attention to the context. Contexts also become important if they are given predictive or informational value. Studies of instrumental (operant) learning are further consistent with the idea that the context might also influence affordances that support voluntary actions. Context switch effects are not universal, but mainly occur when certain attention and perception processes can come into play.

The present article provides a brief review of the role of context that has emerged in recent studies of associative learning. “Associative learning” is a term that is broadly used to refer to a range of learning phenomena that are studied in humans and other animals. Throughout the past century and the beginning of the present one, classical and instrumental conditioning became essential tools for the study of how organisms learn about their environment, as well as to understand some of the neurobiological mechanisms that underlie it¹. In classical (or Pavlovian) conditioning, animals learn about events that occur in their environment. For example, in Pavlovian fear conditioning, rats might learn that the presentation of an auditory cue (e.g., a tone) predicts the occurrence of an aversive stimulus, such as a mild foot shock. In instrumental (or operant) conditioning, the rat’s own behavior produces the reinforcer. For example, rats might be trained to press a lever to obtain a food pellet reinforcer. As we will illustrate in this paper, similar procedures have recently been used in the human experimental psychology laboratory to study human predictive, diagnostic, and instrumental learning^{2, 3, 4}. Associative learning can also be used as a tool to understand how memory works in both human and nonhuman animals. Traditional list learning has been complemented by learning about the correlation between cues or responses and outcomes, akin to nonhuman animal classical and instrumental conditioning⁵. Most of the research has uncovered striking similarities between basic learning processes in human and nonhuman animals (for instance, compare ⁶ with ⁷).

One insight about associative learning is that it never takes place in a vacuum-- it always takes place in the presence of background stimuli or contextual cues. A goal of many studies of human and nonhuman associative learning has therefore been to understand what role, if any, the context plays in directing learning and/or performance. Such contexts have been defined in multiple ways. For example, Bouton⁸ suggested that contexts can include the room or apparatus in which learning takes place⁹, internal states produced by drugs¹⁰, hormone levels¹¹, body temperature^{12, 13}, deprivation state¹⁴, mood state¹⁵, event expectancies¹⁶, recently-experienced events¹⁷, cognitive instructions⁵, or even stimuli that correlate with the passage of time¹⁸. Although room and apparatus contexts have perhaps been studied most extensively, all may follow similar rules. The variety of possible contexts makes it nearly impossible to find a specific definition of what a context actually is. Consistent with this, Smith¹⁹ defined context very generally as “that which surrounds” the target task with which the organism is confronted.

One straightforward idea is that, if the context is important in associative learning, then a context switch should impair test performance on the target task, for example, by preventing complete generalization between the conditions of learning and testing. If a context switch changes performance, then initial learning is said to be context-specific, and performance would be under the influence of contextual control. Although contexts may also be involved in learning in other ways, we will focus our review on those situations in which context change can affect performance, given the historical role of such situations in testing the idea that contexts influence memory retrieval. The idea behind the context switch effect is in fact partly rooted in a tradition from the human memory literature that suggests that memory retrieval depends on the match between the testing and learning background conditions, so that whenever a mismatch occurs, retrieval is impaired^{20, 21}. However, this statement needs to be qualified. The fact is, context switches often fail to produce a change in memory or performance. In what follows, we will describe associative learning research with both human and nonhuman animals that attempts to identify the conditions under which learning appears to be context-specific and the conditions under which it does not. The first sections are mainly devoted to presenting results found in classical conditioning and human predictive learning, though some research in instrumental learning is also discussed. The final section is devoted to instrumental conditioning, as recent research suggests that the context’s role in operant conditioning may have some features that may contrast with those that characterize classical conditioning and predictive learning.

Failures and successes in finding the context switch effect

Let us begin by emphasizing the fact that context change often fails to influence learning and performance. To illustrate, consider a fear conditioning experiment conducted in rats by Bouton and King²². These investigators paired a target tone with a shock in one context (Context A). In the next phase of the experiment, the tone was presented alone (in extinction) in the same context or in a different context (Context B) for separate groups of rats. Presentation of the tone alone, without shock, served to reduce the amount of fear that the tone elicited. Interestingly, though, there was no difference in the amount of fear responding or in the rate of extinction in the two contexts, suggesting that the tone-shock association initially learned in Context A transferred more or less perfectly to Context B. However, when responding to the tone was later tested back in Context A, rats that had been extinguished in B showed strong recovery of conditioned responding (a phenomenon called the renewal effect). This last effect demonstrated that the lack of context-switch effect during extinction was not due to the animals’ inability to discriminate between the contexts (see also ²³). In the end, extinction was strongly context-specific, while conditioning was considerably less so. Similar results have been reported in many other animal conditioning studies and in human predictive learning tasks (see below).

It is interesting to observe that research on human memory has similarly shown many failures of context switches to impair performance^{24, 25}. In a recent example, Cairney, Durrant, Musgrove and Lewis²⁶ gave participants word lists in two different contexts (rooms). Participants were then tested in the same room where learning took place or in a different one, either immediately or after a 12-hr delay. Interestingly, no context-switch effects were reported at the immediate test; word lists were well remembered across different contexts. (There was an effect of context-switch on performance, but only when there was a delay that was spent in wakefulness, rather than in sleep.) The common failure to find context switch effects in human memory experiments contrasts with an older literature on paired-associate learning, where context switch effects were widely documented after an interfering word list was learned in a second phase^{27, 28} (see ²⁴ for a review). After an interference-learning treatment, the context may be important, but in the absence of one, it may be less so.

Studies like these have suggested that first-learned information may not be as context-specific as conflicting information that is learned subsequently^{29, 30}. That is, information learned before extinction, or another retroactive interference treatment, can often be retrieved, with no detectable impairment, in a context where it was not initially acquired. It should be noted that this is not a universal finding. For instance, Bonardi, Honey, and Hall³¹ reported a deleterious effect of context change when a multi-trial acquisition procedure was used (see also ³²). And in other well-known studies, Godden and Baddeley³³ and Smith³⁴ also found that a simple (first-learned) word list was recalled less well after a context change. However, it seems clear that context-specificity of first learned information is not always easy to find²⁵.

Conditions that allow contextual control

While some information appears to transfer quite well across contexts (such as first-learned information, as described above) other information does not. Extinction is probably the best studied example of this. The renewal effect described above has been reported with different animal species (rats³⁵, pigeons³², humans⁵) and a wide range of different tasks such as conditioned taste aversion⁶, conditioned emotional response³⁵, operant behavior³⁶, human discriminative punishment³⁷, human predictive learning⁵, and appetitive Pavlovian conditioning^{38, 39}. The combined results of research on renewal suggest that extinction and, in general, second-learned information about a cue or a response is more easily affected by context-changes than first-learned information or simple acquisition^{30, 40, 41}.

One important feature of the beginning of extinction is that it entails surprise, or prediction error⁴². That is, in the first phase of conditioning, the animal has learned that the CS is associated with a US—but this unexpectedly changes as extinction begins. Surprise is also present at the beginning of a conditioning or learning treatment, when the organism is presented with CSs and USs before it knows anything about the CS-US relation. It is therefore interesting to note that retrieval of simple, initial acquisition also seems to be context specific when testing occurs at early stages of training. For instance, León, Abad and Rosas⁴³ trained human participants in a situation in which they had to predict whether a patron that ate a specific food (the cue) at a given restaurant (the context) would develop a gastric discomfort (the outcome). A context change between training and testing led to a decrease in performance when the change was conducted after 4 training trials, but not when the training was increased to 18 trials (see also ⁴⁴). Similar results had been previously shown with rats. Hall and Honey⁴⁵ reported better transfer of conditioned fear across contexts when rats received 24 trials of conditioning, than when they received a single conditioning trial. Thus, in most cases subjects may tend to code the information as context-dependent at the beginning of training, a dependence that disappears when training is increased (cf. ³¹). Since prediction error is strong at the beginning of training, the situation might be ambiguous and participants are reluctant to discard contexts as potentially relevant for the task. Once they have had the opportunity to learn that contexts are irrelevant (through extended training or through context pre-exposure), context-switch effects disappear.

There are some conditions in which information that is not usually context-specific (e.g., cue-outcome associations in multi-trial procedures) may become context-specific. For instance, Preston, Dickinson, and Mackintosh⁴⁶ (Experiment 2) used a discriminated operant procedure to train two groups of rats to discriminate between two cues in Context A. Responding was reinforced in the presence of one stimulus and not reinforced in the presence of another. This discrimination was kept the same in Context B for one group (Discrimination), but it was reversed for the other group (Conditional Discrimination). Thus, in the latter group Contexts A and B were uniquely informative about the reinforcement contingencies active in the stimuli. Responding in the presence of a new target stimulus was then reinforced in Context A in both groups, and then tested in extinction in both Contexts A and B. Interestingly, there was poor transfer of responding to the target in Context B for the Conditional Discrimination group, while there were no differences in the Discrimination group. Similar results have been reported by León, Abad, and Rosas⁴⁷ in human predictive and human instrumental learning, respectively (see also ⁴⁸). Gawronski, Rydell, Vervliet, and De Houwer⁴⁹ have also reported compatible results in a task in which humans made context-based evaluations of a hypothetical actor’s positive and negative attributes. Thus, giving the context predictive or information value encourages subsequent context-specific learning.

The above results are consistent with the idea that attention to the contexts may play an important role in creating context-specificity: information may become context-specific when the organism has been given a reason to attend to the context^{50, 41}. In perhaps a related way, extinction of a cue-outcome relationship seems to facilitate context-dependence of other simple new cue-outcome associations that are learned at the same time, or after, the first one is extinguished. For instance, in a human predictive learning task, Rosas and Callejas-Aguilera⁵ found a deleterious effect of a context switch on predictive judgments about a cue-outcome relationship when that relationship was learned concurrently with the extinction of a different cue (see also ⁵¹). They also found that extinction of a cue conducted within a particular task facilitated context-specificity of a different cue that was later trained within a slightly different task. Rosas and Callejas-Aguilera⁵² extended these results to Pavlovian learning in rats, finding that extinction of a taste aversion facilitated context-dependence of a different flavor-illness association that was conditioned subsequently (cf. ⁵²). More recently, Bernal-Gamboa, Callejas-Aguilera, Nieto, and Rosas⁵⁴ found that CS-US and response-outcome associations are more “forgettable” over time if there has been previous exposure to extinction of a different association, even within a different task. That result suggests that changes in the temporal context may play a role that is similar to the one played by changes in physical contexts.

In explaining this type of result, Rosas, Callejas-Aguilera et al.⁵⁰ elaborated on Bouton’s⁴¹ suggestion that the ambiguity of a CS’s or response’s association with a reinforcer produced by extinction leads subjects to pay attention to the context. They proposed that once participants pay attention to the context, all the information learned within them becomes context-specific. There has been discussion about whether it is ambiguity or interference that makes information context-specific^{55, 56}. Ultimately, however, when participants encounter prediction error, they appear to behave as if they encode contextual information with concurrently- or subsequently-learned information.

Attentional mechanisms of contextual control

To summarize, research with human and animal subjects strongly suggests that retrieval of information may become context-specific if the circumstances lead to attention to the context. The specific mechanism through which attention operates has yet to be determined, although models of associative learning that incorporate attention processes provide important insights. For instance, Pearce and Hall⁵⁷ suggested that when there is prediction error (i.e., when presentation or omission of the US is surprising), attention will increase to any stimulus that is present on the learning trial. The model thus suggests that extinction will potentially increase attention to the context, as will trials at the very beginning of training (when prediction error is also high), but that the context will become relatively ignored once training is complete^{43, 45} (when prediction error has become low). Alternatively, Mackintosh⁵⁸ argued that when prediction error is low, the contemporaneous stimuli are good predictors, and that attention should be directed toward such good predictors. Subjects will therefore pay attention to predictive and informative contexts^{46, 47}. It is interesting to note that, with a few reasonable assumptions, either of these contrasting approaches can explain results that seem to favor the other. Pearce and Hall⁵⁷ can explain attention to informative contexts if it is assumed that training did not reach the asymptote, so that there is still prediction error. And Mackintosh⁵⁸ can explain why more extensive training or conditioning reduces contextual control because this model predicts that contexts will become ignored when the cue is found to be a better predictor of the outcome. Interestingly, newer models have begun to incorporate both attention rules^{59, 60}.

None of the studies that have been discussed were designed to discriminate among the different approaches to attention in associative learning. And there are other findings that will require further analysis. For instance, Harris, Jones, Bailey, and Westbrook⁶¹ found that extinction leads the extinguished CS to be coded (retrospectively) with the context in which it was conditioned so that a first learned CS-US association becomes context dependent after extinction. It is not clear how the attention mechanisms just described would explain how a remembered context (as opposed to a contemporaneous or to-be-presented context) can be the target of increased attention. One possibility is that extinction might boost attention to the context so that it is processed more at the time of testing.

Contextual control of instrumental conditioning

The preceding sections have emphasized the role of context in classical conditioning and human predictive learning. Does it have a similar role in instrumental conditioning? The question is worth asking, because the operant lever pressing response in rats is psychology’s major animal model of voluntary behavior in humans. (Although lever pressing is controlled by an obvious consequence, the pellet reinforcer, the rat is free to emit it at any time.) The Bouton laboratory has recently looked more systematically at contextual control in operant conditioning, and research to date suggests many important parallels with Pavlovian learning. In particular, there is now excellent evidence of renewal effects ^{36, 62}. For example, in several experiments reported by Bouton et al.³⁶, rats learned to lever press for food pellets on a variable interval schedule of reinforcement in Context A. After responding was extinguished (by no longer presenting the pellets) in either Contexts A or B, responding was renewed when it was tested in a different context. That is, when responding was extinguished in Context B, it recovered when testing occurred in either Context A (ABA renewal) or Context C (ABC renewal); when responding was extinguished in Context A, it recovered when testing occurred in Context B (AAB renewal).

More recent research suggests that all three forms of renewal also occur when the contexts are equated on their reinforcement histories⁶³. For example, in one experiment, rats learned to press a lever or pull a chain suspended from the ceiling for food reward in Contexts A and B, respectively (the responses were counterbalanced so that “R1” was reinforced in Context A and “R2” was reinforced in Context B). In an ABA condition, R1 and R2 were then extinguished in the opposite context and finally tested in the original context. In an AAB condition, R1 and R2 were extinguished in their original contexts and tested in the “new” one. AAB and ABA renewal both occurred. Using an analogous procedure, Todd⁶³ has also demonstrated ABC renewal. AAB and ABC renewal are especially important theoretically, because they indicate that conditioning generalizes better to a new context than extinction does, as is consistent with Pavlovian conditioning and human predictive learning.

However, another result that was observed over all the experiments was that acquisition itself was strikingly context-dependent. Thus, when the experimental design allowed us to compare the rate of operant responding in the context in which the response had been learned (Context A) and in a different context (Context B), we always found a partial, but highly significant, loss of performance when the behavior was first tested in Context B^{36, 63}. Unlike Pavlovian responding, operant responding did not transfer perfectly between contexts. The reason for the difference at first seemed obvious. We had used “free operant” methods, in which the animals were free to lever press or chain pull for reinforcers whenever they chose to. In the free operant method there is no explicit signal, analogous to a CS or predictive cue, that bears a proximate relation to the response. The context is thus the only stimulus, whereas in a Pavlovian method, the context is present along with the CS, which could readily overshadow⁴² or “outshine”²⁵ its control. Bouton, Todd, and León⁶⁴ therefore also studied contextual control using discriminated operant procedures in which lever pressing or chain pulling were only reinforced in the presence of an explicit light or tone discriminative stimulus (SD). With this method, the animal learns to respond only in the presence of the proximate signal. We have observed renewal effects with this method. But more importantly, when the rats learned to respond in the presence of the stimulus in Context A, there was still a drop in performance when the stimulus and the response were tested for the first time in Context B. The drop occurred whether or not a second SD and a second response had been trained in Context B. With our methods, the operant response itself is more context-dependent than the action of the SD.

The experiments thus begin to suggest that, at least in nonhuman animals, operant learning may be relatively context-dependent (cf. ^{46, 62, 65}). One way to think of the result is to note that, with either free-operant or discriminated-operant training, the animal forms a direct association between the response and the context in which it is learned. In the correct context, the response therefore occurs; in the wrong one, it does not. We⁶⁴ further suggested that one can tentatively think of the context-response association the way Gibson⁶⁶ might have: During training, the animal learns that the context “affords” a particular response, just as we learn that a chair can support our body weight or that a hallway can be traversed. Note that there is a sense in which an affordance stands for an action-outcome relationship; it tells us that we can sit on the chair without crashing to the ground or walk upon the floor without falling through. In a similar way, the animal may thus learn that actions on a lever or a chain produce a pellet reinforcer. Regardless of the approach, our preliminary evidence on the effects of context on voluntary, operant behavior suggests that contexts may have surprisingly privileged control over them. Further research will need to extend our understanding of contextual control of operant behavior in nonhuman animals, and ask whether it is found in humans as well.

Conclusions

Our brief review has emphasized several points. One of the most important is that context switches do not always have much impact on Pavlovian responding, predictive judgments in humans, or human list memory. We find, however, that contextual control can emerge with extinction, or a second phase in which conflicting learning is encoded. It also appears to emerge early in training, or when contexts have been given predictive or information value. All of these results are consistent with the idea that context plays a role primarily when experience encourages attention to it. For example, the finding that extinction (or other retroactive interference treatments) creates context-dependence is consistent with the idea that surprise and prediction error can direct attention to the context^{57, 58, 59, 60}.

The case for instrumental or operant learning is consistent, although there is at least one interesting difference. While the research suggests that contextual control is again especially important after extinction, which is consistent with a role for attention, there are indications that instrumental, voluntary actions might be inherently under contextual control—the context-switch effect is more ubiquitous there. Perhaps the operant learning procedure somehow directs the organism’s attention to the context. After all, under most conditions, the lever is a static feature of the context. Or, as we suggested in the preceding section, the context plays a more central role in supporting voluntary action because operant behaviors are supported by knowledge of affordances. Context switch effects in associative learning are not universal, but can occur when certain attention and perception (e.g., affordance) processes are engaged.