The functions of contexts in associative learning

Abstract

Although contexts play many roles during training and also during testing, over the last four decades theories of learning have predominantly focused on one or the other of two families of functions served by contexts. In this selective review, we summarize recent data concerning these two functions and their interrelationship. The first function is similar to that of discrete cues, and allows contexts to elicit conditioned responses and compete with discrete events for behavioral control. The second function is modulatory, and similar to that of discrete occasion setters in that in this role contexts do not elicit conditioned responses by themselves, but rather modulate instrumental responding or responding to Pavlovian cues. We first present evidence for these two functions, and then suggest that the spacing of trials, amount of training, and contiguity are three determinants of the degree to which the context will play each function. We also conclude that these two functions are not mutually exclusive, and that future research would benefit from identifying the conditions under which their functions dominate behavioral control. We close by discussing some misconceptions concerning contexts.

1. Introduction

Traditionally, the analysis of the associative structure underlying both Pavlovian and instrumental conditioning has focused on discrete events such as briefly presented cues that are paired with reinforcers (i.e., Pavlovian conditioning), or responses that are followed by reinforcers (i.e., instrumental learning). Although the analysis that follows does incorporate Pavlovian events, responses, and instrumental reinforcers, we will focus on contexts, or environments. By definition, contexts are a complex array of stimuli extended in space and time, and these stimuli can emanate from external (outside world) and/or internal sources (the internal state of an animal). In the laboratory, contexts are operationally defined depending on the particular task being used, and the gamut of operational definitions is wide. For example, the chamber or apparatus where experiments using rats, mice, pigeons and monkeys are conducted, are typically considered the environment or context in which learning takes place. Experiments concerned with spatial learning use local cues (e.g., cues within the boundaries of a maze) and distal arrays as stimulus attributes that animals (Morris, 1981) and humans (Doeller & Burgess, 2008; Doeller, King & Burgess, 2008) use to successfully guide their behavior in space. Effects of altering or changing contextual attributes have also been documented when the subject’s internal state is manipulated by either the administration of drugs (e.g., Overton, 1964; 1985) or the induction of emotional states (e.g., Bower, 1981).

As can be seen, there are a multitude of different factors that appear to collectively constitute the context or environment in which learning/retrieval takes place, and the issue becomes more complicated when considering work with humans, where these operational definitions can be extended to abstract dimensions. For example, contextual control of behavior and thought can be achieved by instructing subjects that an event to be remembered took place in a particular location (Callejas-Aguilera & Rosas, 2010; Orinstein, Urcelay, & Miller, 2010), or by the semantic attributes of the to be remembered items, as has been documented in verbal learning (Tulving & Thomson, 1973). However, the constituents of context are not limited to the conventional sensory modalities. For example, Bower (1981) has demonstrated that affective state can serve as a component of contextual control of memory, and Bouton (1993; 2010) has proposed that the current time can act like an attribute of context (e.g., a stimulus experienced today can be treated as somewhat different than the same stimulus if it is experienced tomorrow). By equating time and space, Bouton’s proposal successfully accounts for changes in memory performance that are brought by changes in either physical or temporal attributes of the environment in which learning and/or retrieval takes place. Of course there is an asymmetry between spatial and temporal components of context in that one can return to a prior spatial context but not to a prior temporal context, at least absolute temporal context as relative temporal contexts can be recreated (i.e., 5 s following onset of a red circle).

The difficulties in operationally defining what constitutes the context have also permeated theoretical debates, with different theories assuming different roles for contexts during learning and during retrieval. We as well as others have argued that contexts may play fundamentally different roles depending on different training and testing circumstances (Balsam, 1985; Bouton, 2010; Holland & Bouton, 1999; Miller & Schachtman, 1985; Rudy, 2009). The present review constitutes an update of this literature in which we summarize some of the results produced in the last 25 years and attempt to refine our concepts and ideas about the many roles that contexts play. In particular, we believe that it is important to identify the circumstances that favor each of the possible ways in which contexts can influence acquired behavior. One possibility is that contexts differentially influence learned behavior depending on whether one assesses the role of context on what is immediately being learned or the role of context on immediate performance (Miller & Schachtman, 1985), although as far as the animal is concerned, surely every trial is both a learning trial and an opportunity to perform. In fact, the situation is made even more complex by the fact that memory retrieval is an active process that influences subsequent learning (Arnold & McDermott, 2013; Miller, 1982). Rather than assessing the functions of contexts in terms of training versus retrieval, in this review we focus on different functions that emerge with the use of different parameters in the task, such as the spacing of trials, the relationship between stimulus duration to overall context exposure (i.e., the C/T ratio; Gibbon & Balsam, 1981), and the role of contiguity. We emphasize these variables because research has shown that they can strongly influence the effect that the context or environment has in memory performance, and it can either impair or facilitate depending on parametric variations along these dimensions.

In an attempt to reduce conceptual complexity and redundancy, we have identified two functions of contexts that seem to permeate different literatures and experiments using different preparations and species, which is not to imply that there are not additional functions of contexts. We will describe these two functions in terms of the operations used to differentiate between them, and we will use this as a starting point for reviewing our recent work on the roles of context. First, the context can act as a memory modulator (or occasion setter; Bouton & Swartzentruber, 1986), a role that is demonstrated by differences in responding to a discrete conditioned stimulus (CS) as a function of testing in the context in which that CS was trained as opposed to another context with the same associative history (but with a CS other than the target CS) and equal familiarity. Second, contexts can act as cues (CSs), which is best demonstrated by changes in responding to a CS as a function of the test context having been extinguished (i.e., posttraining exposure to the context in the absence of the unconditioned stimulus [US]) or associatively inflated (i.e., posttraining context-US pairings), compared to no posttraining manipulation of the associative status of the context. This assay is based on the widely held assumption that such associative deflation and inflation does not change the modulatory potential of the context (Holland, 1992). With few exceptions, theories of learning have traditionally adopted one of these two functions for contexts, but surprisingly there is no formalized theory that accounts for both roles, nor one that specifies the parameters under which one or the other of these roles will be best exposed. Our recent research has implications for this explanatory gap as it attempts to specify the conditions under which each function will be revealed (which is not to imply that the two roles of context are mutually exclusive), and it also sheds light on how these different functions interact. These two roles are the focus of this brief review.

So far we have circumscribed our discussion to two families of functions for contexts. Here we attempt to better characterize these. The context can act as a cue, which can interact with the target cue (i.e., a discrete stimulus) during training and can elicit behavior on its own. Conditioned responding to the context can summate with responding to the target cue during testing. The former property permits contexts to compete with discrete stimuli for behavioral control. For example, in fear conditioning, contexts in which animals have received a mild footshock will elicit a freezing response, which is understood as reflecting fear to the context (e.g., Fanselow, 1980). When discrete CSs are tested in such contexts, fear to the context summates with fear to a discrete CS (e.g., Balaz et al., 1981, 1982; Polack, Laborda, & Miller, 2013). Contexts acting as cues may also interact with discrete stimuli by entering into competition with these cues, as is most evident when training trials are massed (e.g., Barela, 1999; Miguez, Witnauer, Laborda, & Miller, 2014), or when USs alone are administered before (e.g., Randich & Ross, 1984) or during conditioning (e.g., Rescorla, 1968; Miguez, Witnauer, & Miller, 2012; Urcelay & Miller, 2006). In some circumstances, the cue-like properties of a context as a direct predictor of a US and as a cue that competes with discrete CSs may interact (Polack et al., 2013). Finally, by establishing a negative contingency between a particular context and the absence of shock, context can also acquire conditioned inhibitory properties evidenced by summation and retardation tests (a canonical pair of procedures to assess conditioned inhibition; Rescorla, 1969; Polack, Laborda, & Miller, 2012). In one way or another, many models have assumed this ‘cue’ function for contexts (e.g., Gallistel & Gibbon, 2000; Gibbon & Balsam, 1981; Harris, 2006; LePelley, 2004; Mackintosh, 1975; McLaren & Mackintosh, 2000; Miller & Matzel, 1988; McClelland & Rumelhart, 1985; Pearce, 1987; Pearce & Hall, 1980; Rescorla & Wagner, 1972; Stout & Miller, 2007; Sutton & Barto, 1988; Wagner, 1981).

Alternatively, the context may act as a retrieval modulator (i.e., an occasion setter). This role of context takes the form of the context (or its absence in contrast to its presence) modulating at test the retrieval of US representations given presentation of a nominal CS (e.g., Bouton & Swartzentruber, 1986; Miguez, Cham, & Miller, 2012; Molet, Urcelay, Miguez, & Miller, 2010). Evidence for this role is abundant as there are many instances in which the presence of a specific context at test is essential for the expression of CS-US associations acquired within that context (e.g., Urcelay & Miller, 2008). The context dependency of excitatory learning has been demonstrated compellingly in memory interference paradigms with humans (for an early review, see Underwood, 1969) and in other experiments investigating memory retrieval (e.g., Godden & Baddeley, 1975) in addition to the above mentioned state-dependent retrieval (e.g., Bower, 1981; Overton, 1985). For similar effects in animal studies, see for example Amundson and Miller (2008) and Miguez, Cham, et al. (2012). Although excitatory learning is often context dependent, extinction learning (a specific memory interference paradigm) seems to be even more so, with its retrieval at test being critically dependent on the presence of spatial and temporal cues from extinction training (Bouton & Bolles, 1979; reviewed in Bouton, 2004; Laborda & Miller, 2012; Urcelay, 2012). Thus, given that these effects are readily observed in several different preparations and species, [some] theories of learning and memory have embraced this function and consequently been able to explain a wealth of observations (e.g., Bouton, 1993; Miller & Escobar, 2002; Spear, 1973; Tulving & Thomson, 1972).

The dual function of contexts that we have described is slightly perplexing given the important role that contexts or environments have in cognition and acquired behavior. On the one hand, many theories capture contexts as cues, but have difficulties in explaining a number of context modulation effects such as recovery from extinction and, more generally, memory interference effects. On the other hand, theories of learning that view contexts as facilitators for the retrieval of information can account for these interference effects, but fail to account for effects such as US preexposure or trial massing, both of which are context dependent and presumably are mediated by context-US associations (Urcelay & Miller, 2008). In the last few years we have conducted a number of experiments designed to identify which variables influence the engagement of each of these two functions. The guiding principle appears to be that animals are opportunistic information processors and ordinarily use all available information in near optimal ways (Tolman & Honzik, 1930). This invites the question of how different conditions, such as low contingency, low contiguity, and massed training (captured as a low C/T ratio – see below) between discrete cues and outcomes, determine how contexts will function to hamper learning or at least the behavioral manifestation of learning. It is important to note that all these conditions lead to ‘ambiguity’ (in a rather general sense of the term) regarding the putative CS-US association. For example, ambiguity can be created by training circumstances in which the CS or US are presented alone (low contingency), when CS-US pairings are massed thus facilitating the context acting as a competitor, or simply by separating in time CS-US presentations (low contiguity).

1.1 Situations that lead to ambiguity

Low contingency situations are characterized by unpaired presentations of either the cue or the outcome, or both, and these can be administered before or after cue-outcome pairings or interspersed between cue-outcome pairings (e.g., Miguez, Witnauer, & Miller, 2012). Context dependency of acquired responding is often observed when cue- or outcome-alone presentations are conducted before or after cue-outcome pairing (i.e., situations that introduce ambiguity; e.g., Miller & Escobar, 2002). The classic example of this is extinction learning, in which a cue is first paired with the outcome (acquisition) and subsequently presented alone (extinction). Extinction learning is widely regarded as acquisition of a new Cue-NoOutcome association that is highly context dependent, and a similar context dependency is observed when cue alone exposures occur before training (i.e., latent inhibition; Lubow & Moore, 1959) or when the outcome is presented alone either before or after cue-outcome pairings (Randich & LoLordo, 1979; Urushihara, Wheeler & Miller, 2004; for boundary conditions on when outcome-alone presentations following cue-outcome pairings will degrade behavioral control by the cue, see Miller & Matute, 1996). Similarly, in proactive and retroactive associative cue interference, different CSs are paired with the same US across two phases (or a common CS is paired with two different outcomes across two phases), and a decrement in responding based on the target association (either the first learned or second learned association) is observed due to training on the alternative (interfering) association (e.g., Amundson, Escobar, & Miller, 2003; Miguez, Cham, et al., 2012).

One way to summarize this literature is as follows: either when two cues share the same outcome, or when two different outcomes follow one cue, interference is observed at least with select parameters (see Miller & Escobar, 2002, for discussion), and these effects are highly dependent on the similarity of the test context to the training context(s), both in terms of spatial and temporal attributes of the test context. Even in situations in which the successive phases of training do not differentiate the two interacting associations, as in the cases of partial reinforcement (i.e., unsignaled CSs interspersed among the CS-US pairings) and degraded contingency (i.e., unsignaled USs interspersed among the CS-US pairings), information retrieval has been shown to depend on the context presumably because such an ambiguity directs attention to the context (Rosas et al., 2006). Indeed, Rosas et al.’s account of modulation of acquired information by the training context differs significantly from the more conventional views of Bouton (e.g., 1993) concerning outcome interference and of Miller and Escobar (2002) concerning both outcome and cue interference, in that Rosas et al. contend that anything which directs attention to the context (e.g., ambiguous reinforcement history and instructions to attend to the context) increases context dependency of any information subsequently acquired in that context.

Less clear, however, are the conditions necessary for memories to be context dependent in cases such as continuous reinforcement in which ambiguity is not provided by the training conditions and attention to the context is not encouraged by other means such as instruction. Here the literature is mixed, with most reports in Pavlovian conditioning failing to find context dependence in memory, some finding a decrement (perhaps greatest under conditions of few CS-US pairings; Honey & Hall, 1990) and some actually finding facilitation of performance after a context change, although only in aversive preparations (Kaye & Mackintosh, 1990). Continuous reinforcement, however, can also be conducted under conditions of high ambiguity, in particular when the context as well as the discrete CS accurately predict the US, which encourages the context serving as a cue for the US and consequently competing with the discrete CS. For example, when CS-US trials are massed, the context becomes a good a predictor of the US, in which case it is likely to compete for behavioral control with the discrete CS. Note that massed training is not only operationalized by short intertrial intervals (ITIs), but critically by the ratio of CS exposure to context exposure, a phenomenon well captured by timing theories (e.g., Gibbon & Balsam, 1980). In other words, a low ratio of context exposure to CS exposure (achieved by either short ITIs, long CSs, or both) will facilitate the context becoming a good predictor of the US and competing with discrete cues.

Another phenomenon that fosters the context’s functioning as a cue for the US and competing with discrete cues for responses is trace conditioning, which refers to situations in which an interval is introduced between CS termination and US onset (Balsam, 1984). It is notable though that the circumstances regarding the context dependency of trace conditioning, to the best of our knowledge, have not yet been detailed. Indeed there seems to be a dearth of evidence regarding these questions. For example, given Pavlovian trace conditioning, is responding to the target CS context dependent? Does responding to a CS increase if, after excitatory training, the training context undergoes extinction? This is remarkable given that trace conditioning depends heavily on the hippocampus, which presumably mediates responding to contextual cues (Marchand, Luck, & Di Scala, 2004). Where evidence is available, we will review such studies as they may illuminate the conditions under which different functions of the context are engaged.

The context dependency of extinction (and interference paradigms in general) has been widely documented and thus it will not be the main focus of this review (see Bouton, 1993; Miller & Escobar, 2002; Urcelay, 2012, for reviews). In the remainder of this article, we will review recent and earlier data suggesting that a low ratio of CS exposure time (C) to context exposure time (T) facilitates the operation of the context as a cue for the US, and that this function, although dissociable from the modulatory role of context, is not mutually exclusive of the modulatory function but rather can coexist with it. That is, we suggest that a low C/T ratio will favor the context competing for behavioral control with discrete CSs, but not necessarily decreasing the potential modulatory role of the context. Critically, in addition to low C/T ratio, we will argue that low contiguity of the discrete CS (i.e., long intervals between CS termination and US onset as in trace conditioning) favors the context coming to function like a cue in predicting the US, and hence acquiring the potential to compete with discrete CSs for behavioral control.

2. The role of the spacing of trials (and C/T)

Our initial goal in the following experiments was to determine whether the cueing and modulatory functions of contexts are dissociable from each other. To answer this question, we needed a preparation in which both contextual functions would occur. We capitalized on two phenomena that, although they had been studied in separate literatures, are procedurally similar but produce opposing results. Indeed, together they provide a paradoxical scenario. When CS-US pairings are preceded by US-alone exposures, context mediation of the CS-US association is inferred from the observation that if the unsignaled USs are presented in a context different from that of CS-US training, the typical deficit in responding to the CS (i.e., the US preexposure effect) is severely attenuated (e.g., Tomie, 1976). Thus, a number of theories of learning have assumed that context-US associations produce the deficit in acquisition (or at least expression) of the CS-US association, and consequently predict that the US-preexposure effect should be most readily observed when both US-preexposure and acquisition trials are massed because massed trials would minimize extinction of the context between trials. Indeed, theories of learning further predict that if the USs of US preexposure treatment are immediately preceded by a second [nontarget] CS, the effect should also be attenuated because the second CS should overshadow the context. This prediction also applies when the US-alone presentations are intermixed with CS-US pairings (i.e., degraded contingency treatment) and again has received empirical support in Pavlovian (e.g., Durlach, 1983) and instrumental learning (Dickinson & Charnock, 1985).

The US pre-exposure effect stands in sharp contrast with a phenomenon in the memory interference literature (Underwood, 1969) known as proactive associative interference between cues (e.g., Amundson, Escobar, & Miller, 2003). In proactive cue interference, CS1-US pairings prior to CS2-US pairings proactively interfere with performance reflecting the CS2-US pairings, and this effect is clearly context dependent (Amundson & Miller, 2008) as is the US-preexposure effect. Interference effects are thought to depend on the association between CS1 and the US, so that if CS1 and US presentations during phase 1 are unpaired, or nonexistent, the deficit in responding to CS2 is not observed. What stands in sharp contrast with proactive cue interference is that, in the US-preexposure effect, signaling phase 1 USs with CS1 increases responding to CS2, whereas in proactive interference, signaling phase 1 USs with CS1 produces a deficit in responding to CS2 (i.e., a proactive cue interference effect). In other words, signaling USs during phase 1 results in an increase (US preexposure) or a decrease (proactive interference) on CS2 performance. Based on the assumption that contexts are more prone to function as cues when trials are relatively massed and learning about CS1 should be superior (allowing CS1 to overshadow the context) when phase 1 trials are spaced, we sought to document these two opposing effects as a function of trial spacing during phase 1. We used a factorial design in which phase 1 US exposures were administered under massed (in 10-min sessions) or spaced (in 120-min sessions) conditions. Orthogonally, groups received these US exposures either unsignaled or signaled by a nontarget cue. We expected, and observed, a cross-over interaction without main effects, which indeed documents the inverse effects of trial spacing upon signaled as opposed to unsignaled Phase 1 USs. That is, when phase 1 trials were spaced, signaling US-alone presentations with CS1 resulted in decreased behavioral control by CS2. But when phase 1 trials were massed, signaling US-alone exposures resulted in increased behavioral control by CS2 (Urcelay & Miller, 2010, Exp. 1). This cross-over interaction is difficult for contemporary models of learning to explain because these theories have been designed to account for one or the other effect of the Phase 1 signal, but not both.

Having established parameters that reliably yield the US-preexposure effect (i.e., massed unsignaled US presentations in phase 1) and proactive cue interference (i.e., spaced signaled US presentations in phase 1), we next sought to determine whether the cueing function of a context presumably engaged with massed trials during phase 1 and the modulatory function presumably engaged with spaced trials during phase 1 are doubly dissociable. The underlying reasoning was that if these two functions are truly independent, one could manipulate one effect but not the other, and vice-versa. To achieve this goal, two experiments were conducted, each using the exact parameters that in the aforementioned experiment resulted in reliable US-preexposure and proactive interference effects. In one experiment (Urcelay & Miller, 2010, Exp. 2), we massively pre-exposed subjects to the context in order to produce latent inhibition of the context (Cole et al., 1996; Lubow & Moore, 1959). We observed that latent inhibition of the context selectively attenuated the US-preexposure effect, but left the proactive interference effect intact. Thus, we found one manipulation (i.e., latent inhibition) that selectively affected the context when it was presumably serving as a cue, but had no effect upon the context when it was serving as a modulator, consistent with the notion that stimulus [context here] preexposure (i.e., latent inhibition treatment) does not interfere with that stimulus subsequently becoming an occasion setter (Oberling, Gunther, & Miller, 1999). Indeed, as has been suggested for occasion setters, the modulatory function that the context seems to play in proactive interference does not appear to depend on the context’s direct associative strength; otherwise latent inhibition of the context should have impaired the ability of the context to modulate proactive interference.

The third experiment in this series was aimed at assessing whether we could affect the modulatory function of the context without changing the cueing function that was affected after context preexposure. There is substantial evidence that when phases 1 and 2 of training take place in different physical contexts, the relative similarity of the test context to the two training contexts strongly influences the extent to which proactive interference is observed (Amundson, Escobar & Miller, 2003; Amundson & Miller, 2008). Assuming, as some researchers have (e.g., Bouton, 1993), that the passage of time can result in a change in contextual attributes, we sought to determine whether proactive interference decreases with time (i.e., a waning recency effect following phase 1), as has been reported in other tasks (e.g., Wright et al., 1985). Across both the human and nonhuman memory literatures, there seems to be a consensus that in associative interference situations behavioral control depends on how immediately recent the two learning episodes were relative to test. That is, in interference situations the two pieces of conflicting information apparently coexist in memory, and initial easy retrievability of recent information gives way over increasing retention intervals to retrieval of information learned at an earlier stage. This constitutes the so-called recency-to-primacy shift (e.g., Postman, Stark, & Fraser, 1968). Shifts from recency to primacy can be accommodated by relatively simple rules that weight conflicting information as a function of the age of the information (e.g., Devenport, 1998). Proactive cue interference involves ambiguous information about the predictors of an outcome (CS1-US vs. CS2-US) and because these associations are trained sequentially, we hypothesized that if time is interpolated between training the CS1 and CS2, proactive interference should wane. That is, proactive interference depends on the temporal [and physical] context at test modulating interference based on the test’s relative recency to CS1-US training and CS2-US training. Thus, interpolating a retention interval between phases 1 and 2 should favor information acquired during phase 2, thereby attenuating proactive interference. In contrast, there is little evidence suggesting that the US preexposure effect, an effect that depends on the context as a cue rather than as a modulator, would change with the interval between preexposure in phase 1 and target training in phase 2. Fear memories concerning context are known to last for months (Witgen & Silva, 2007), and in fact previous research from our laboratory has suggested and an 8-day retention interval does not affect the strength of the US-preexposure effect (Matzel, Brown, & Miller, 1987). In our last experiment (Urcelay & Miller, 2010, Exp. 3), we again used parameters that had previously yielded proactive cue interference and US-preexposure effects, but we interposed either a 1- or 15-day retention interval between phases 1 and 2 of training. As expected, proactive interference was greatest when phases 1 and 2 occurred with a short interval between the two phases of training (i.e., in the same temporal context), and was reduced by a change in temporal context; however, blocking by the context (as a cue) did not vary with the interval between phases.

These findings can be summarized as follows. The contexts in which learning takes place and can be later tested can have at least two different functions and the impact of each of these functions are highly dependent on the parameters employed. This is merely a description as here we are not describing the mechanism through which contexts have these diverse functions, but our experiments indicate that seemingly minor variations in training procedures can have strong effects on memory performance and no contemporary theory of learning successfully anticipates all of our observations. The second conclusion that stems from this series is that these two functions of context are dissociable, in that we could manipulate each function separately without affecting the alternative function. Perhaps the important question facing us, now that we can dissect these different functions, has to do with the interaction between the two functions. By interaction, we are asking whether cueing and modulatory functions can be implemented simultaneously or whether contexts can at one time fulfill only one function or the other. The second series of experiments that we present here addressed this question.

3. The role of context with extended training in Pavlovian conditioning

Since early observations by Pavlov (1927), and later by others (Kamin, 1961; Millenson & Hendry, 1967), there have been a number of reports that, with extended reinforced training, conditioned responding paradoxically decreases after peaking (i.e., response vigor as a function of number of CS-US pairings is an inverted U-shaped function). We have previously referred to this decrease in behavioral control as the postpeak performance deficit (PPD). The source of this decrease in responding with extended reinforcement training is not clear, as some researchers have argued that animals adapt or habituate to repeated presentations of the US and thus decrease responding to the CS (Pavlov, 1927; Kamin, 1961) and others have proposed that this adaptation is context dependent (Bouton et al., 2008). In a sense, Bouton et al.’s account combines a US habituation mechanism with an additional constraint to this adaptation; it is context dependence. Thus, Bouton et al.’s explanation is similar to the modulatory function of contexts that we isolated in the above-mentioned series (Urcelay & Miller, 2010), a conclusion that was bolstered by the observation that PPD wanes when a retention interval is imposed between the end of training and testing (Pickens et al., 2009). In contrast to this explanation, theories of learning focused on retrieval processes such as the comparator hypothesis (Miller & Matzel, 1988) account for the PPD by assuming that the context, serving as a cue, progressively gains associative strength with extended training, allowing it to better compete with the target cue for retrieval at the time of test. Presumably context continues to acquire associative strength with the US after the discrete cue has reached an asymptote because the context has a lower salience than the discrete cue. Moreover, memory of the training context is thought to negatively interact with discrete cues at the time of test (Stout & Miller, 2007; also see Witnauer, Urcelay, & Miller, 2013, for simulations). In summary, the various explanations of the PPD all emphasize a role for the context, but differ in terms of what this role is, with some arguing that it reflects a modulatory function whereas others argue for a cueing function.

We recently undertook a series of experiments with the aim of differentiating among these accounts of the PPD, while also illuminating any potential interaction between these functions (Urcelay, Witnauer, & Miller, 2012). In a first experiment, we simply arranged training so that across an equal number of days (5) four groups of rats would receive 5, 10, 20, or 50 pairings of a 30-s clicker and a mild footshock. The spacing of trials was equated across groups, so that differences in responding could not be due to this variable (see above C/T). After five days of training, all groups were tested for lick suppression (testing in this experiment was conducted in a different [neutral] context in order to preclude associative summation between the CS and test context confounding interpretation of the CS lick suppression data). The results replicated previous findings; responding was maximal after 5 or 10 conditioning trials, lower (albeit not significantly lower) after 20 trials, and appreciably decreased after 50 trials, which documents yet again the PPD phenomenon (Urcelay, Witnauer, & Miller, 2012; Exp. 1).

In a subsequent experiment (Urcelay et al., 2012, Exp. 4), we tested a prediction made by the comparator hypothesis (and its successor the sometimes competing retrieval model [SOCR]; Stout & Miller, 2007), namely that post-training extinction of the training context should increase responding to the CS significantly more after 50 training trials than after 5 training trials. In other words, if the decrement observed after 50 trials is due to competition by the context with the target CS at the time of testing, then extinction of the context between training and test should alleviate the PPD. We tested this prediction using a 2×2 factorial design in which subjects received 5 or 50 conditioning trials, with the same parameters as we used in the previous experiment. After training, subjects were orthogonally assigned to a condition in which the context was extinguished for 10 hours or a control condition that in total received 20 min of context exposure. All animals were tested on the CS in a neutral context three days after completion of the context extinction manipulation. The results revealed an increase of responding after context extinction. As predicted by the extended comparator hypothesis, the increase was larger after 50 training trials than after 5 training trials, resulting in an interaction between amount of training and context extinction. This experiment suggests that, at least with the parameters employed, the context had a function akin to that of a discrete cue in forming a direct association with the outcome (although that was not directly measured in these experiments because testing was conducted outside of the training context), and importantly that it competed for control with the target CS such that context extinction resulted in changes (an increase) in behavioral control by the discrete CS (see Polack, Laborda, & Miller, 2013, for corroborating data).

The experiments just summarized suggest that one candidate explanation for PPD is that the context, acting as a second cue, competes for control with the discrete cue. But the experiments described so far do not address the modulatory function described by Bouton et al. (2008) nor do they speak to the potential interaction between cueing and modulatory functions of a context. In fact, the results just described seem to be at odds with the report by Bouton and colleagues (also see, Pickens et al., 2009). In particular, we note that in our experiments testing was always conducted in a neutral context in order to prevent fear to the context summating with that of the cue being tested. Contrary to our observation of PPD outside of the training context, Bouton and colleagues (2008; also see Pickens et al., 2009) observed that PPD was attenuated when testing was conducted outside of the training context. Thus, the contradiction is that we observed PPD outside of the training context, whereas they observed [partial] recovery from PPD outside of the training context. To resolve this apparent paradox, we undertook a parametric analysis of the reports by Bouton et al., and Pickens et al. This revealed that, consistent with our earlier analysis (Urcelay & Miller, 2010), the overall amount of context exposure that they used was much longer than that used in our previous experiments. In addition, they both used bar-press suppression preparations, which required at least six sessions of magazine and bar-press acquisition in the training context to establish a baseline and thus measure fear suppression. As described above, massive context preexposure can attenuate the context’s potential to act as cue because it retards development of an effective context-US association (i.e., latent inhibition of the context). Indeed, as described above, latent inhibition of the context attenuated the US-preexposure effect (Urcelay & Miller, 2010), while leaving modulatory functions of the context unaltered (as was the case in proactive interference). Thus, it is plausible that the discrepancies between these reports and our data are due to differences in the parameters employed, and assessment of this possibility was the aim of the following experiment.

To address whether the differences between our results and those of other labs were due to differences in the spacing of trials, which in turn affected how the training context functions, we conducted an experiment focused on PPD (i.e., all subjects received extended training) and varied the spacing of the training trials (Urcelay et al., 2012, Exp 2). That is, all subjects received ten daily sessions of training, alternating between two contexts with a different CS in each context (all counterbalanced). Half of the subjects received this training with a spacing of trials similar to that of our previous experiments (hereafter massed), whereas the remaining half of the animals received similar training but with a spacing of trials similar to that of Bouton et al., (2008) and Pickens et al., (2009). In this experiment, subjects were tested twice, one time with each of the trained CSs. One CS was tested in the same context in which it was trained, whereas the other CS was tested in a context different from that in which it had been trained, with the order of testing counterbalanced across subjects within groups. The results revealed a cross-over interaction in the absence of main effects. In the condition in which trials were relatively massed during training, stimulus control by the CS was stronger in the training context than in the alternative context. This could be attributed to massed training establishing the context as a competing stimulus for the discrete cue, and summating at the time of test (see Polack et al., 2013), but this is not likely as both contexts (same or different) received similar amounts of training, and with similar ITIs in each subject. Thus, the massed condition, despite using parameters that resulted in context competition (as suggested by the context extinction data, see above), also revealed positive modulation by the context that cannot be attributed to differences between the contexts in associative strength to the US (also see Rescorla, 2008). The group that received spaced trials, using parameters similar to those of other labs, revealed the opposite pattern. Stimulus control by the CS was weaker in the context in which training occurred, and stronger when testing occurred outside of the training context. This pattern replicated that of other laboratories, and thus established the spacing of trials (the length of the ITI) as the (or at least one) critical variable determining which function, or both, will control behavior. In addition, this design equated context-US associations within each trial-spacing condition, so the difference in responding within the spaced trial condition reflects a modulatory role of the context in the absence CS ambiguity such as in an interference paradigm. This suggests that CS ambiguity is not necessary for contextual control, a finding that is consistent with at least some reports in the human (e,g., Godden & Baddeley, 1975) and nonhuman memory literature (e.g., Hall & Honey, 1990).

Together with the results of the previously described series (Urcelay & Miller, 2010), the present PPD experiments (Urcelay, Witnauer, & Miller, 2012) provide strong support for the view that trial spacing is a critical determinant of the functional properties of the context. We first supported this assertion with a double dissociation of these two functions of the context, based on the view that short ITIs (or a low C/T) favor the context controlling behavior as a cue, and that long ITIs favor the context controlling behavior as a modulator. In addition, the second set of experiments extended these results to the PPD effect, and further showed that these two functions, although dissociable, are not mutually exclusive. This conclusion stems from the fact that with relatively massed training trials, which should favor the context becoming functionally similar to a cue, we still observed modulatory effects. Responding was higher in the training context than elsewhere, even when summation of the context-US association with target CS-US association was controlled by equating the context-US association across the context in which the target CS was trained and the context in which the target CS was not trained.

4. The role of contiguity

Contiguity has traditionally been proposed to be a critical determinant of associative learning; early theories of learning assumed that contiguity was necessary and sufficient for learning to occur. This was the case until observations that, despite perfect contiguity, behavioral control sometimes did not occur. Examples of these are Rescorla’s observation that decreasing the contingency between CS and US, by presenting either (or both) alone, had detrimental consequences for behavioral control by the CS, despite CS-US contiguity being optimal (Rescorla, 1968). Contemporary to Rescorla’s observation, Kamin documented the phenomenon of blocking, which refers to decreased behavioral control by a cue as a consequence of its being trained in the presence of an already established predictor of the US (Kamin, 1968; also see overshadowing, Pavlov, 1927). Because a blocked cue is typically trained with optimal CS-US contiguity and fails to control behavior, some learning theorists abandoned contiguity as necessary and sufficient for learning to occur, and focused instead on what has been referred to as ‘informational variables’ (i.e., contingency). We have performed a brief and selective review of the role of contiguity in behavior elicited by contexts and cues, and found some illuminating relationships which we briefly summarize below.

In instrumental learning, specific behaviors are followed by reinforcement and thus, the probability of the emission of such behaviors increases. Contiguity plays a role in instrumental behavior, as decreases in instrumental behavior are typically observed when delays are imposed between target responses and reinforcement, a finding that has been observed in both rats (e.g., Cheung & Cardinal, 2005; Dickinson, Watt, & Griffiths, 1992) and humans (e.g., Shanks & Dickinson, 1991). An issue here is what role, if any, response-reinforcer contiguity plays in the way in which contexts influence instrumental behavior. As we described above, it is possible to assess the cueing role of the context by administering context extinction after instrumental training sessions, and see what effects, if any, extinction has on the target instrumental behavior. Such a manipulation was performed by Pearce and Hall (1979). In their study, rats were trained to lever press for food reinforcement on a VI 60-s schedule of immediate reinforcement. When instrumental behavior was stable, a group of rats received context exposure, which should have led to extinction of context-US associations, and a second group received context exposure but with ‘free’ presentations of food pellets during the context exposure sessions (which should have prevented the extinction achieved by context exposure). A third group did not receive context exposure and remained in their home cages. During a subsequent test of instrumental responding in the experimental context (in the absence of reinforcement), subjects that received context exposure (i.e., extinction) pressed less vigorously than those that received exposure but with free pellets delivered during the exposure session, or those that received no context exposure. In other words, extinction of the context decreased instrumental behavior suggesting that, when reinforcement is immediate, contexts invigorate instrumental behavior. However, a finding in sharp contrast to that which we just described was reported by Reed and Reilly (1990). Similar to Pearce and Hall, they trained rats to press on a VI 60-s schedule of reinforcement. However, instead of delivering the reinforcers immediately after the reinforced lever press, they delayed the reinforcer for 6 seconds. After instrumental behavior was stable, they administered context extinction. In contrast to what was found by Pearce and Hall, they observed that after context extinction, rats pressed more vigorously during a test in which reinforcement was withheld. This result was soon replicated and extended to a wider range of parameters by Dickinson, Watt, and Griffiths (1992). They found in rats that were trained with a 64-s delay of reinforcement, notably a delay that does not ordinarily support acquisition of free operant behavior, that context exposure in the absence of lever and reinforcers increased instrumental behavior (the context exposure was done interspersed among the instrumental acquisition sessions). Even with a 64-s delay of reinforcement, rats were able to acquire instrumental behavior which became manifest as a result of this context extinction treatment. These results are consistent with those of Cheung and Cardinal (2005) who observed that hippocampal lesions had opposite effects in instrumental acquisition depending on whether the reinforcer was presented immediately after lever pressing, or delayed by 10 or 20 s. That is, they conducted complete (or sham) hippocampal lesions before they administered instrumental training with 0-, 10-, and 20-s delays of reinforcement. Relative to their sham controls, hippocampal rats showed retarded instrumental responding when the reinforcer was presented immediately (0-s delay; a result similar to that of Pearce & Hall, 1978, after context extinction), but stronger responding when the reinforcer was delayed for 20 s (consistent with the context extinction results of Reed & Reilly, 1990, and Dickinson et al., 1992). Overall, assuming that hippocampal lesions disrupt context processing, these results resemble those obtained when the context was extinguished.

Taken together, these data suggest that when reinforcement is immediately contiguous to the instrumental response, an excitatory test context invigorates instrumental behavior, but when reinforcement is delayed, the ambiguity introduced by the delay in reinforcement changes the facilitative function to the test context into a competitive one. It is only then that context extinction prior to testing in that context results in an increase in instrumental behavior, just like the increase we observed when context extinction followed PPD training in a Pavlovian procedure (Urcelay et al., 2012). In fact, similar results have been observed in Pavlovian fear conditioning, but using trace conditioning procedures. Marlin (1981) observed strong freezing to a fear CS when shock was presented with a 0-s trace, but less freezing when the trace interval was 10- or 30-s. Notably, preference for the context in which the CS was trained showed the opposite pattern, in that preference for the training context, which is inversely related to fear to the context (Odling-Smee, 1978), was greater in the group trained with a 0-s trace, and lesser in the group trained with a 30-s trace. Finally, a recent report assessed delay (0 trace) and 30-s trace fear conditioning and varied the spacing of trials during training (Detert, Kampa, & Moyer, 2008). Consistent with the instrumental data, they observed increased trace conditioning (30-s trace) when trials were spaced than when they were massed, and this difference was inverted when they measured context fear, suggesting that context conditioning follows a pattern inverse to CS conditioning when a trace is imposed between these Pavlovian events. Thus, this relationship appears to occur in both Pavlovian and instrumental procedures.

5. Functions of contexts

The notion that there are two roles for contexts in Pavlovian (and instrumental) learning is clearly an oversimplification, and it is likely the case that each of the two types of roles of context is more complex than we have characterized it here (Rudy, 2009). For example, test contexts as cues can either add or subtract from responding to the target CS. Associative summation of contexts and CSs has been demonstrated extensively in the excitatory domain (e.g., Balaz et al., 1981, 1982; Polack, Laborda, & Miller, 2013), and recently it has been shown that under certain conditions a context in which extinction occurs can acquire inhibitory properties, ultimately passing summation and retardation tests for inhibition (Polack, Laborda, & Miller, 2012). Thus, the presence of an excitatory or inhibitory context can add or subtract to responding to a discrete CS, following relatively well established principles of associative summation.

However, training contexts can also modify responding to a discrete CS even when the training context is not present, a process which is central to the comparator hypothesis (Miller & Matzel, 1988; Stout & Miller, 2007). The comparator hypothesis posits that a comparator stimulus which became associated with the target cue during training will reduce responding to the target CS at test provided there is a strong comparator stimulus-US association, even when the comparator stimulus is not present at test. A layman’s description of the critical process according to the comparator hypothesis is as follows. During training, an animal encodes three critical associations, a CS-US association, a CS-context association, and a context-US association (because they are all presented together). When the animal is presented at test with the CS, it expects that the US will soon occur based on the CS-US association, but it will also consider other predictors of the US that were present during training of the CS (i.e., comparator stimuli) such as the context, so that even an absent context will reduce responding to the CS (because it is associatively retrieved) provided that CS-context and context-US associations are strong. Basically, the comparator hypothesis asserts that animals respond not in accord with the extent to which the CS predicts the US, but the extent that the CS predicts an increase in the likelihood of the US relative the likelihood of the US based on other cues that were present during training of the CS. A similar assumption is made by timing models (Gallistel & Gibbon, 2000; Gibbon & Balsam, 1981). In summary, when parameters favor the context functioning as a cue, the presence (associative summation and comparator process) or absence (comparator process) of the context at test can be highly influential on responding to a discrete CS.

As we have mentioned throughout this paper, contexts as modulators can also play opposing roles, either increasing or decreasing responding to the target CS. There are several examples of this modulatory function. Occasion setting refers to the property of a stimulus, whether it is a discrete or diffuse like a context, by which the stimulus does not itself elicit a conditioned response, but rather modulates responding to a discrete CS. The literature on occasion setting is extensive and examples include both positive and negative occasion setting (Holland, 1992). Contexts, in terms of their modulatory role, are seemingly similar to the discrete occasion setters studied extensively by Holland and others in that, despite their not eliciting conditioned responding on their own, they still increase (positive occasion setting) or decrease (negative occasion setting) responding to a discrete cue that is presented at test with the putative occasion setter. A number of characteristics lend support to this parallel, which was first made by Bouton and Swartzentruber (1986). For example, discrete stimuli of low salience and extended duration act better as occasion setters than salient and short duration stimuli, and these characteristics (diffuse) resemble the physical properties of contexts, in that they are present throughout a learning session or experience. These analogies have encouraged configural explanations of context-stimulus interactions that suggest that animals process context and the cues presented in them in an unitary fashion (Pearce, 1987). Although this explanation has some success in explaining a number of these interactions, it does not fare well with the occasion setting data (see Holland, 1992, for a discussion)

The modulatory function of contexts is also similar to that of a discriminative stimulus in an operant preparation, in that it signals whether a given response will be reinforced, and this elicits execution of that response in a choice situation. Given the different analogies established between the modulatory function in Pavlovian and instrumental conditioning, one clear conclusion is that the different modulatory roles of contexts are currently poorly understood (see Molet et al., 2010, for an example of modulation both during learning and test). Nevertheless, it should be evident from the present discussion that contexts are capable of both (a) acting as cues that enter in competition with discrete CSs (or instrumental responses) and elicit conditioned responding, and (b) signaling whether a CS (or response) will be followed by reinforcement independent of any direct context-US association. Given these independent functions of context and the fact that cueing and modulatory functions of a context can coexist, it is important to establish which variables determine the relative strengths of cueing and modulatory functions. In the next section, we review evidence suggesting that, in addition to trial spacing, a second variable, contiguity, can strongly influence whether the context will act as a cue for, or a modulator of acquired behavior.

Overall, the present review suggests that the spacing of trials and a trace between antecedent and consequent events both affect how contexts influence acquired behavior, but in opposing ways. Because these results are problematic for all theories of learning, we suggest that these relationships should be incorporated into future theorizing. We have identified two variables that affect context processing, but this by no means suggests that these are the only two variables that do this. Likely there are other variables that affect how contexts are processed, and we believe that a future line of inquiry illuminating these would be fruitful, for it would further our understanding concerning the roles played by contexts in acquired behavior, and incorporate these roles into our models of learning.

6. Some misconceptions about contexts and environments

Since the publication of the seminal book “The hippocampus as a cognitive map” by John O’Keefe and Lynn Nadel (1978), a large literature has appeared suggesting that spatial information is mediated by the hippocampus, and specifically that the hippocampus contains a ‘module’ in which spatial information is processed. The evidence supporting this assertion is abundant; the hippocampus seems to be critical for spatial learning (Moser & Moser, 1998) as much as it is critical for context processing in fear conditioning (Selden, Everitt, Jarrard, & Robbins, 1991). In fact, Selden and collaborators were first in describing a double dissociation between hippocampus and amygdala in terms of context and cue processing, respectively. This dissociation supports the view that there are distinct modules in the brain that process different sources of information (Cheng, 1986; Gallistel, 1990; but see Cheng, 2008), with the hippocampus being critical for spatial and context processing in classical conditioning (Fanselow, 2010). A consequence of this thinking is that geometric (i.e., spatial) information and contextual processing are thought to be unique in the way they are processed (Fanselow, 2010). Two sources of evidence seem to support this claim. First, geometric information seems to be immune to cue competition effects such as blocking and overshadowing (Doeller & Burguess, 2008a; 2008b), and experiments in spatial learning sometimes find the opposite of overshadowing, that is, potentiation of geometry learning by landmarks (Pearce et al., 2006).

Some authors have proposed that learning rules involving geometry should be different than those governing discrete stimuli, and devised models which are specific to spatial learning in predicting potentiation (e.g., Miller & Shettleworth, 2007). Our results and the reviewed data on parametric variations suggest that these assertions only apply to one end of the parametric spectrum, and challenge the view that contexts are unique in terms of processing. In fact, potentiation was first described in taste aversion learning (Clarke, Westbrook, & Irwin, 1979; Galef & Osborne, 1978), which led John Garcia to claim that taste aversion learning required a specific module that functioned with different laws from those employed for processing sensory stimuli of other modalities (Garcia, Lasiter, Bermudez-Rattoni, & Deems, 1985). Perhaps the question should be whether potentiation is specific to taste and geometry learning, and the answer seems to be ‘no’. A few years ago, we reported experiments in fear conditioning in which we were able to obtain overshadowing with two discrete auditory cues, a loud tone and a soft clicker, when shock was presented immediately after termination of the compound cue. However, when shock onset was delayed 20 s after termination of the compound cue, the overshadowing effect reversed into potentiation, suggesting that contiguity is a critical determinant of whether overshadowing or potentiation will be observed (Urcelay & Miller, 2009; also see Batsell et al., 2012, for a replication in taste aversion, and Sissons, Urcelay, & Miller, 2009, for an overshadowing-to-potentiation reversal achieved by varying CS duration, i.e., C/T). Our (Urcelay & Miller) 2009 studies were first to suggest that an environmental variable (contiguity) can determine whether cue competition or facilitation (i.e., overshadowing or potentiation) will be observed, suggesting that there is no need to assume special modules for spatial or taste learning.

A second argument favoring the uniqueness of contexts has been proposed by Fanselow, following the observation of the so-called immediate shock deficit. The immediate shock deficit is the observation that better context fear learning is observed when there is a greater time between placement in a novel environment and shock administration. Empirically, rats seem to require about 30 s of context exposure before the shock onset for effective conditioning of fear as assessed a day later. Because this finding seems at odds with the observation that shorter discrete CSs condition better than longer CSs, Fanselow proposed that the slope of the function between amount of exposure and maximum conditioned responding (see Fanselow, 2010; Fig 1) is negative for conditioning of discrete cues and positive for conditioning of contexts. This is based on the view that contexts must be processed as a configuration (or “Gestalt”) that requires a period of time for the configuration to be completed (Fanselow, 2010; also see Miller, 1970). Our results (Urcelay & Miller, 2010; Exp 2) suggest that with sufficiently long preexposure impaired behavioral control by the context occurs rather than the improved context learning reported by Fanselow. Thus, there appears to be an inverted U-shaped function for contextual control of responding as a function of the interval between CS and US presentations. In fact, a proper assessment of the CS-US function (see Rescorla, 1988; Fig 1), suggest no grounds for this assertion. Indeed, it is likely that the function, when properly evaluated, has the form of an inverted U and this seems to be the case for both contexts and cues. Thus, there is nothing special about context in terms of spatial learning and exposure/responding functions, rather these results seem to be found under extreme parametric variations that, when properly assessed, show that both context and cues display similar functions (Barnet, Grahame, & Miller, 1993a; 1993b; 1995).

7. Concluding remarks

Contexts, understood as those environmental cues that surround a learning experience, have a strong influence on learning and expression of past events. We have presented evidence that parametric variations in terms of CS duration to context duration ratio (C/T) and contiguity have a strong effect on what is learned, and critically dissociate two different (but not mutually exclusive) functions of contexts. Whereas recent reviews have emphasized only one function of the context (e.g., Maren, Phan, & Liberzon, 2013) or claimed that contexts are unique (Fanselow, 2010), we suggest that the processing of contexts is not unique and that its two functions are not necessarily mutually exclusive. These results may have important implications for the processes engaged in relapse after exposure-based therapies (Urcelay, 2012), and raise the question of which circumstances are critical for relapse to occur. Future theories of learning should attempt to accommodate these findings if they want to make clear and testable predictions concerning the influence of context on acquired behavior. The variables we have identified are not the only ones, but at least they go some way in shedding light on the factors that determine the dominance of one function or the other in specific situations.

Highlights.

• The two families of functions of context in associative learning are cueing of an outcome and modulating the expression of cue-outcome associations.

• These two functions are not mutually exclusive.

• The extent of the cueing and modulatory roles of context are strongly influenced by trial spacing, amount of training, and contiguity.