Occasion Setting

Abstract

Occasion setting refers to the ability of one stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning.

Advances in understanding the neurobiology of learning and memory have followed innovations in both technology and the exploitation of useful conceptual distinctions and experimental procedures from behavioral psychology. Study of the neurobiology of fear and reward learning has especially benefited from the analysis of brain mechanisms of phenomena examined extensively by behavior theorists, such as stimulus selection (for example, blocking and overshadowing), extinction (for example, spontaneous recovery, reinstatement, and renewal), incentive learning (for example, conditioned reinforcement and Pavlovian-instrumental transfer), evidence for representation of reinforcer properties in learning (for example, devaluation and differential outcome expectancy tasks), and the learning of conditional discriminations (for example, patterned and biconditional discriminations). Much of this analysis has been guided by ‘elemental’ theories such as the Rescorla-Wagner (1972) model, which make simple assumptions about the functions of conditioned stimuli (CSs) and how those CSs combine in learning and action. For example, CSs are typically assumed to elicit conditioned responses (CRs) and/or activate representations of unconditioned stimuli (USs). Similarly, CSs are often thought to compete for limited amounts of learning, and their associative strengths assumed to add linearly to form aggregate predictions. But for decades psychologists have recognized that more complex stimulus functions and combination rules are important in all but the simplest learning situations.

Consider a rat in a simple “feature positive” (FP) discrimination learning task in which a target cue (T) is paired with food when it is accompanied by another, feature cue (F), but not when it is presented alone (FT+, T−). Most elemental theories, such as the Rescorla-Wagner model (1972), suggest that the rat would come to attribute food delivery entirely to the feature. Configural theories, such as those of Pearce (1987, 1994, 2002) suggest instead that the rat parses the task as comprising two stimuli, a feature+target configuration that predicts food, and a solitary target that predicts nothing. A third option is that the rat attributes food delivery to the target, but learns that the feature distinguishes between the occasions on which the target is followed by food and when it is not (Holland, 1983).

Here, we review considerable behavioral evidence that this last, “occasion setting” strategy is a frequent contributor to animal and human discrimination learning, and suggest that some clinical problems, such as persistent relapse in addiction and post-traumatic stress disorder, may be better modeled by occasion setting than by simpler associative processes. We first distinguish occasion setting from simple conditioning and describe several assays of occasion setting. Next, we describe the conditions under which occasion setting is acquired, the content of that learning, and major theories of occasion setting. In the course of this discussion we compare occasion setting and accounts for configural learning. Finally, we review some initial attempts to explore the neurobiology of occasion setting, and argue that the time is ripe for exploiting new neuroscientific techniques to advance that study, which may enhance our understanding of a range of learning processes.

Distinguishing occasion setting from simple conditioning

When investigating any instance of learning, it is important to consider both its conditions and its consequences. That is, we must determine what circumstances enable and influence its acquisition, and how the organism is changed by that experience. The conditions of learning may include environmental factors such as the nature of the stimuli and the spatial and temporal relations arranged among them, and organismic factors such as internal states. The consequences of learning may include its representational structure, such as associations or networks, and properties or functions acquired by the stimuli involved, such as the ability to elicit CRs or serve incentive functions.

Evidence from several laboratories suggests many distinctions between simple conditioning and occasion setting, in both the circumstances under which they occur and their consequences (e.g., Bonardi, Robinson, & Jennings, 2017; Holland, 1992; Schmajuk, Lamoureux, & Holland, 1998; Swartzentruber, 1995). Initial studies of the circumstances and consequences of occasion setting were intertwined and relied on appreciable bootstrapping. Hypotheses about the representational structure of occasion setting guided the selection of procedural variables that might encourage it, allowing the identification of acquired properties and functions unique to occasion setting. Development of these assays for occasion setting then made possible the more detailed identification of circumstances that favor the occurrence of occasion setting, and led to refinement in understanding of its representational structure.

Assays of Occasion setting

Although it was informed by earlier work (e.g., Moore, Newman, & Glasgow, 1969; Skinner, 1938), modern study of occasion setting began with an examination of how rats solved Pavlovian feature positive (FT+, T−) and feature negative (FT−, T+) discriminations. Within simple elemental conditioning models, animals would learn that the feature signals the upcoming occurrence or nonoccurrence of the US. As excitatory or inhibitory feature-US associations are formed, the features acquire the ability to activate or inhibit the activation of a representation of the US, eliciting (Figure 1a) or suppressing (Figure 1c) responding. A working hypothesis for occasion setting is that animals learn that the feature (an “occasion setter”) signals whether the relation between the target and the US is active or not (Figures 1b and 1d). Thus, the occasion setter acquires the ability to modulate or gate a target-US association, and hence the target’s ability to activate the US representation.

Figure 1. Schematic of associative structures of serial or simultaneous feature training.

Associative structures established in simultaneous (panels A and C) and serial (panels B and D) feature positive (panels A and B) and feature negative (panels C and D) discriminations. F and T refer to representations of feature and target stimuli, respectively, US refers to a representation of the unconditioned stimulus, and X refers to a representation of a transfer test stimulus paired with the US outside of the discrimination task (see sections on transfer effects in “Assays of occasion setting” and “Contents of Occasion setting”). Arrows indicate excitatory associations and bars indicate inhibitory associations. Filled circles indicate positive modulatory (occasion setting) links and open circles indicate negative modulatory links. The dotted arrows refer to excitatory associations established after counterconditioning of the feature (see sections on counterconditioning in “Assays of occasion setting” and “Contents of Occasion setting”).

Early research developed three assays to distinguish occasion setting from simple elemental conditioning: differences in response form, differences in the effects of extinction or counterconditioning, and differences in transfer functions. This research was guided by the intuition that occasion setting would be encouraged when the feature and target were presented serially on compound trials (we will elaborate on this intuition later). We mostly describe results from a single conditioning preparation, food-reinforced Pavlovian learning in rats, but note evidence from other preparations and species in passing.

Response form.

An obvious difference between simple conditioning and occasion setting accounts of FP discrimination learning is their specification of the stimulus that elicits behavior when the feature+target compound is present. The conditioning account assumes that responding to the compound is the consequence of feature-US associations, whereas within an occasion setting account, responding to the compound is the consequence of target-US associations, which are gated or enabled by the occasion setting feature. Ross and Holland (1981) identified the associative origins of responding in FP discriminations by using a food-reinforced conditioning preparation in which the form of the CR is partly determined by the nature of the CS (Holland, 1977, 1984b). For example, with certain visual cues paired with food, rats rear on their hind legs at cue onset and then stand quietly with their heads in the food cup until food delivery, but with some auditory cues they exhibit a startle response to stimulus onset, followed by short, rapid “head-jerk” movements, usually in the vicinity of the food cup. These behaviors are clearly differentiable, do not simply reflect performance effects, and occur as the consequence of the Pavlovian CS-US contingencies (see Holland, 1984b for a review). For our purposes, a valuable characteristic of this preparation is that the form of conditioned responding during a light+tone compound paired with food reveals its associative origin. That is, if the CR comprises rearing and quiet food cup behaviors, then it is the consequence of light-food associations, but if the CR to the compound comprises startle and head jerk behavior, it is the consequence of tone-food associations.

Ross and Holland (1981) found that when the feature and target cues were presented simultaneously in FP discriminations, the form of the CR acquired to the feature+target compound was characteristic of the feature that predicted reward. When a light+tone compound was reinforced and the tone target alone was nonreinforced, the tone alone elicited no behavior and the light+tone compound evoked rear and food cup behavior, as did presentations of the light feature alone in probe tests. Similarly, if the light was used as the target and the tone as the feature stimulus (i.e., light+tone→food, light→ O), the light alone elicited no behavior, and both the light+tone compound and the tone alone elicited startle and head- jerk. These observations are consistent with elemental conditioning theories, which attribute solution of FP discriminations to the formation of feature-US associations.

However, when the feature preceded the target on compound trials, very different patterns of behavior emerged. For example, within a serial light → tone→food, tone→nothing FP discrimination, although the light once again elicited rearing, the tone elicited head-jerk behavior (characteristic of auditory CSs) when it was presented after the light, but no behavior when it was presented alone. Similarly, when a tone→light compound was reinforced and the light alone was nonreinforced, the rats acquired both head jerk and startle during the tone feature and rear behavior during the light, but only on compound trials. Thus, in addition to behavior occurring as the consequence of feature-US associations, the target cues also controlled behavior characteristic of target-US associations. Because that responding to the target occurred only on serial compound trials, Ross and Holland (1981) suggested that rats solved serial FP discriminations in part by using the feature to set the occasion for responding based on target-US associations, rather than by simply associating the feature and US, as in simultaneous FP discriminations.

Rescorla (1985) reported a similar pattern within pigeon autoshaping experiments. Although pigeons come to peck localized key lights paired with food delivery, no such pecking occurs to diffuse auditory or visual cues paired with food. However, Rescorla (1985) found that if a diffuse feature cue signaled when a key light was to be reinforced in a FP discrimination, that feature acquired the ability to set the occasion for pecking the lighted key on compound trials, again supporting a distinction between occasion setting and simple excitatory conditioning properties of a feature cue on the basis of response form.

Extinction and Counterconditioning.

If responding to the feature-target compound in FP discriminations reflects simple feature-US associations, repeated presentations of the feature alone after discrimination training was completed should extinguish that responding. Indeed, after simultaneous FP training, nonreinforced feature presentations substantially reduced responding to the feature+target compound, not only with Pavlovian appetitive training like that just described (Holland, 1989b), but also in fear conditioning (Holland & Petrick, unpublished), and discrete-trial operant procedures (Holland, 1991a) in which food was delivered during the feature+target compound in training only if a lever-press occurred. By contrast, after serial FP training, although extinction of the feature again reduced responding attributable to direct feature-food associations (e.g., rearing to the light in a light→tone compound), the feature’s ability to set the occasion for behavior controlled by the target was unaffected in each of the three preparations just described (Holland, 1989b, 1991b; Holland & Petrick, unpublished), as well as in pigeon autoshaping (Rescorla, 1986a) and spatial landmark tasks (Leising, Hall, Wolf, & Ruprechtl, 2015), although in some preparations this result has not been obtained (e.g., Fonteyne & Baeyens, 2011; Mainhard, Parodi, & Rojas, 2008).

In the preceding experiments, nonreinforced feature presentations were massed in an extinction phase after FP training was completed. Analogous results were obtained when those nonreinforced feature presentations were instead intermingled within FP training itself, converting it to a “positive patterning” (Woodbury, 1942) discrimination (FT+, F−, T−). Holland (Holland, 1989a; Ross & Holland, 1981, 1982) showed that adding nonreinforced feature presentations to a serial FP discrimination did not slow (and in some cases enhanced) learning. Within elemental theories of learning, the added nonreinforced feature trials should slow the acquisition of feature-US associations responsible for responding to the feature+target compound. Indeed, adding F- trials to FP discriminations when simultaneous compounds were used dramatically increased the discrimination difficulty.

This immunity of the feature’s occasion setting powers to simple nonreinforcement is consistent with our working hypothesis that occasion setting involves the feature’s modulation of a target-US association, independent of any feature-US associations. Although feature-alone presentations disconfirm feature-US predictions and hence should extinguish CRs based on simple feature-US associations, they provide no information about the target-US relation, and thus should have no effect on responding controlled by that relation.

Analogous immunity of occasion setting to manipulations of the feature-US relations occurs with feature negative (FN) discrimination learning, in which the target stimulus is reinforced when presented alone, but nonreinforced when accompanied by the feature (T+, FT−). In experiments using either food (Holland, 1989d) or electric shock (Holland, 1984a) USs, rats received either serial (T+, F→T−) or simultaneous (T+, FT−) FN training, followed by reinforced feature (F+) presentations (“counterconditioning”). We anticipated that simultaneous training would establish inhibitory feature-US associations, but that serial training would endow the feature with the ability to modulate the target-US association, setting the occasion for nonreinforcement of the target (“negative occasion setting”). As expected, after simultaneous training, counterconditioning of the feature abolished its ability to inhibit responding to the target on feature+target compound trials. Indeed, the excitation acquired to the feature on counterconditioning trials (dotted arrow in Figure 1c) summed with that originally established to the target: after counterconditioning, responding was greater on feature+target trials than on either feature or target trials alone.

However, after serial FN learning, counterconditioning of the feature had relatively little effect on its ability to inhibit responding to the target (Holland, 1984a; 1989d; see also Rescorla, 1985, in pigeon autoshaping, but see Baeyens, Vervliet, Vansteenwegen, Beckers, Hermans, & Eelen, 2004, in a human conditioned suppression experiment). Although responding to the feature and target cues was substantial when each was presented separately, responding to the target was still substantially reduced when it was preceded by the feature (recall that with simultaneous training procedures, after feature counterconditioning, the compound elicited more responding than the individual elements). As with the effects of extinction on occasion setting in FP discriminations, this immunity of negative occasion setting to counterconditioning is consistent with our working hypothesis that occasion setting involves the feature’s modulation of a target-US association: the feature’s excitatory links with a US representation (dotted arrow in Figure 1d) can be independent of its inhibitory links with the target-US association.

Examination of the course of counterconditioning itself gives another illustration of this independence. After simultaneous FN training was completed, acquisition of excitatory feature-US associations was retarded relative to acquisition to a control cue (Holland, 1984a; 1989d), as expected if inhibitory feature-US associations had to be overcome (Figure 1c, a “retardation test” of conditioned inhibition, Rescorla, 1969). However, after serial FN training, acquisition of excitatory feature-US associations was not slowed, because there were no inhibitory feature-US associations to overcome (Figure 1d; Holland 1984a, 1989d): the feature had instead acquired the ability to modulate the excitatory target-US associations.

A final example of the independence of a feature’s occasion setting powers and its direct associations with the US comes from experiments in which reinforced presentations of the feature are intermingled within FN discrimination training. Compared to FN discriminations, solution of this “negative patterning” (F+, T+, FN−) discrimination is notoriously difficult (Whitlow & Wagner, 1972; Woodbury, 1942) when the compound elements are presented simultaneously, and is impossible within simple elemental theories, because the summation of the strengths of the excitatory feature-US and target-US associations requires a greater response to the compound than to either of its elements. However, Holland, Thornton and Ciali (2000) found that when serial feature→target compounds were used, separate reinforcement of the feature enhanced acquisition of negative pattering discriminations relative to feature negative discriminations. Again, the concurrent acquisition of excitatory feature-US associations did not interfere with the acquisition of negative occasion setting to the feature. Holland, et al. (2000) attributed the enhancement observed (as did Rescorla, 1991b, for a similar result in pigeon autoshaping) to increases in attention to the feature. Notably, Holland, et al. (2000) found that lesions of the amygdala CeA, known to eliminate these kinds of attentional enhancements, eliminated the facilitatory effects of feature reinforcement on discrimination learning.

Morell and Holland (1993) reported a particularly strong demonstration of the independence of a feature’s simple conditioning and occasion setting powers. In one condition, rats received two serial negative patterning discriminations with the same target (F1+, T+, F1→T− and F2+, T+, F2→T−) and then compared the effects of F1, F2, and a compound of F1+F2 on responding to T. Although the F1+F2 compound suppressed responding to the target more than either feature alone, it also evoked more responding than either feature alone. Thus, both the negative occasion setting and simple excitatory powers of the features summed, independently, in opposite directions.

Transfer effects.

Within elemental learning theories, a cue’s simple excitatory or inhibitory associative strength is assumed to combine arithmetically with the strengths of other cues paired with that US, because each derives its strength from associations with a representation of that US (Figures 1a and 1c). Thus, within FP discriminations, the feature’s excitatory strength should be revealed whether it is presented alone, in compound with its original target, or with some other cue, say X. Indeed, if X’s strength is also excitatory, we would anticipate the feature+X compound to control more responding than the feature alone; furthermore, if X’s excitation is greater than the residual strength of the original target (which should be low or nonexistent according to those theories), then responding during a feature+X compound should exceed that to the original feature+target compound. Likewise, within a FN discrimination procedure, the feature should reduce responding controlled by its original target or any other excitatory target. In fact, the occurrence of such decremental effects (in a “summation test”) in the latter case is a part of the standard definition of conditioned inhibition (Rescorla, 1969). Many experiments have confirmed these predictions in simultaneous FP and FN training (Holland, 1992).

By contrast, if the occasion setting power of a cue is not dependent on the simple association of that cue with the US, but rather involves the modulation of the target-US association, then an occasion setting feature in a FP or FN discrimination should only modulate responding to its original target cue, absent any stimulus generalization between the original and transfer test targets (Figures 1b and 1d). Many experiments from Holland’s laboratory, using Pavlovian appetitive and fear conditioning procedures as well as discrete-trial operant reward procedures (e.g., Holland, 1986b, 1989a, 1989d, 1991a,b; Holland & Lamarre, 1984; Lamarre & Holland, 1985) showed that occasion setters failed to modulate responding to stimuli that were trained separately, outside of occasion setting procedures. Similar specificity has been observed in other laboratories (e.g., Bonardi, et al., 2017) and preparations, including pigeon autoshaping (Rescorla, 1991a, b) and various human conditioning procedures (Baeyens, Vansteenwegen, Hermans, Vervliet, & Ellen, 2001; Baeyens, et al., 2004; Dibbets, Maes, & Vossen, 2002). All in all, the observation of successful transfer of feature’s excitatory or inhibitory powers after simultaneous FP or FN training, but the failure of serially-trained features’ occasion setting power to transfer to new, separately-trained targets both supported the distinction between simple CSs and occasion setters, and substantiated the claim that occasion setters act on particular target-US associations. We take up a more subtle consideration of transfer effects later when we consider the “Content of Learning in Occasion Setting”.

Conditions for the Establishment of Occasion Setting

The data described in the preceding section show that the serial and simultaneous training procedures generate different learning, and that the serial procedure favors the acquisition of occasion setting. Although not universal, this pattern has been observed in a number of conditioning preparations and species (e.g., the previous section of this article; Baeyens, et al., 2001, 2004; Dibbets, et al., 2002; Nakajima, 1992; Nelson & Bouton, 1997; Young, Johnson, & Wasserman, 2000; but see Rescorla, 1989). In general, different circumstances favor occasion setting and conditioning, and often those that favor one discourage the other. In this section, we first consider some critical temporal features of serial discriminations that encourage the acquisition of occasion setting, and then consider other, nontemporal factors.

Although much of this research comprised systematic exploration of various parameter spaces that differentiated the serial and simultaneous procedures used in Holland’s laboratory, it was mostly guided by two preliminary hypotheses. The first was a casual gestalt notion that occasion setting is encouraged by conditions that produce perceptual discontinuity between the feature and target, that is, which favor grouping of the target and US, and the separation of that target-US unit from the feature. For example, serial presentation of feature and reinforced target might encourage subjects to parse the sequence as feature → (target→US), rather than feature→US or (feature + target) → US. It is notable that this intuition seems opposite to what one might expect for the construction of a unique, configural (feature→target) cue. Thus, research guided by this hypothesis also informs a simple configural alternative to occasion setting, mentioned in the Introduction.

A second hypothesis was that occasion setting occurs when conditions favor more rapid conditioning of the target than of the feature, despite the feature’s being a more valid predictor of the US. In a serial FP procedure, for example, although the target cue is a relatively poor predictor of whether the US will occur (only a portion of target trials is reinforced), it is highly predictive of when the US will occur, because of its close contiguity with the US on reinforced compound trials. Conversely, although the feature consistently predicts reinforcement on a trial, it is relatively noncontiguous with that reinforcer. If this delay to reinforcement substantially slowed the rate of simple conditioning to the feature, then the most efficient strategy of anticipating the US might be to use the feature to identify which trials were reinforced, and the target to determine when reinforcement was to occur. This strategy would be especially likely if occasion setting could operate over longer inter-stimulus intervals (ISIs) than simple conditioning (as will be shown in the next section). These two hypotheses proved useful in specifying many differences in the circumstances that yield occasion setting and simple conditioning, and led to more mechanistic statements about these circumstances.

Temporal factors

The serial and simultaneous procedures used in our early FP and FN occasion setting experiments (Lamarre & Holland, 1984; Ross & Holland, 1981) differed in several ways, including the feature-US, feature-target, and target-US intervals. In a series of experiments with Pavlovian appetitive FP procedures (Holland, 1986a; 1992; Ross & Holland, 1981), we systematically examined the effects of variations in each of these intervals, across a range of intervals and interval combinations. In all of these experiments, the feature cues were visual and the targets were auditory, so that we could index simple conditioning to the feature by rear and quiet food cup behaviors, and occasion setting to that feature by head jerk behavior during the target. Several distinctions were evident. First, although (as in most conditioning preparations) simple conditioning was greater with shorter feature-US intervals than with longer intervals, occasion setting was minimal with those shorter intervals and was substantial over a broad range of longer intervals, including those that supported only minimal simple conditioning. Indeed, the incidence of occasion setting increased over the same range of feature-US intervals that simple feature conditioning decreased. Second, within the parameter space we examined, the critical determinant of occasion setting was the use of longer feature-target intervals, independent of the feature-US or target-US intervals. For each feature-target interval examined, the target-US interval (and hence the feature-US interval) had no significant effect on the acquisition of occasion setting. Again, it is notable that these longer feature-target intervals might be expected to discourage solution of FP and FN discriminations using a configural strategy.

Interestingly, variations in the feature-target intervals not only affected the amount or strength of occasion setting, but also seemed to be part of the content of learning. Holland (Holland, 1998; Holland, Hamlin, & Parsons, 1997) found that target responding was reduced if the target was presented at shorter or longer feature-target intervals than were used in training, suggesting that the feature-target interval was represented in the associative structure of occasion setting. Furthermore, when features trained with different feature-target intervals were combined, the optimal target presentation time suggested a non-algebraic averaging rule, which differed from that found for the timing of simple CRs controlled by those features. By contrast, Bonardi and Jennings (2007) and Nakajima (2009) found that when feature cues signaled different target-US intervals, although the target-US temporal map was modulated by the features, subsequent variations in the feature-target interval did not have consistent effects. Finally, introducing a gap between feature termination and target onset (Holland, 1986; Holland & Ross, 1981) substantially enhanced occasion setting, relative to conditions in which the feature terminated with target onset or target termination (which did not differ). This enhancement contrasts with the “trace conditioning” deficit usually observed when a gap is introduced between CS termination and US delivery in simple conditioning (e.g., Ellison, 1964; Holland, 1980, Thompson, Moyer, & Disterhoft, 1996). Likewise, insertion of a gap between feature and target might be expected to discourage formation of a configural feature+target cue. On the other hand, in pigeon autoshaping procedures, Nakajima (1993a, 1994) found that the introduction of such a gap had mixed effects on solution of serial FP and FN discriminations, and slowed acquisition of a serial ambiguous feature (F→T1+, T1−, F→T2−, T2+) discrimination (Nakajima, 1993a), also thought to involve occasion setting (considered in depth in a later section, “Independence of positive and negative occasion setting”).

The occurrence of occasion setting is also affected by variations in the inter-trial interval (ITI). Casually speaking, within the perceptual discontinuity notion, the more isolated the target-reinforcer pair is from the next feature, the more likely that pair will be coded together, and separate from the feature. Thus, separating serial compound trials by larger ITIs might be anticipated to enhance the acquisition of occasion setting, especially when the feature and target were separated by longer gaps. Experiments using a discrete-trial operant lever press preparation verified both of these predictions in both serial FP (Holland, 1995) and serial FN (Holland & Morell, 1996) discriminations. Not only was discrimination learning itself facilitated in this manner, but so was the proportion of that learning attributable to the use of an occasion setting strategy (rather than to simple conditioning of the feature), as measured by transfer or feature extinction tests. Notably, within most of the parameter space examined, the ITI to feature-target interval ratio was the best predictor of the amounts of occasion setting obtained.

Non-temporal factors

Although temporal variables had major impact on the occurrence of occasion setting in our experiments with rats, they are not the sole determinant. Indeed, as noted below, we have obtained occasion setting with simultaneous compounds under some circumstances, and Rescorla (1989) found occasion setting with both serial and simultaneous compounds in pigeon autoshaping. In this section, we discuss four non-temporal influences on occasion setting: element similarity, intensity, reinforcement history, and context.

Feature-target similarity.

If occasion setting is enhanced by manipulations that place psychological distance between the feature and target, while maintaining the relation between target and reinforcer, then occasion setting should be best when feature and target are most dissimilar. Arranging similarity relations between feature and target would encourage association between those cues, “bridging the temporal gap” between them, and perhaps favoring the configuring of feature and target to form a new, unique cue that could be directly associated with the reinforcer, reducing the likelihood of occasion setting. Indeed, in both Pavlovian appetitive conditioning (Holland, 1989a) and fear conditioning (Lamarre & Holland, 1987) with rats, the use of similar-modality feature and target cues slowed the acquisition of occasion setting in serial FP and FN discriminations. As described in the subsequent “Configural theories” section, the observation that occasion setting is discouraged by the use of similar cues is often seen as a major problem for configural theories of occasion setting accounts. Intuitively, more similar stimuli would seem more likely to be processed as parts of a single configuration than a group of dissimilar stimuli (e.g., Soto, Gershman, & Niv, 2014), and substantial data indicate that the use of similar elements facilitates within-compound learning (e.g., Holland & Ross, 1981; Rescorla, 1986c, Rescorla & Furrow, 1977).

Target intensity.

One of our guiding hypotheses was that occasion setting occurs when conditions favor more rapid conditioning of the target than of the feature, despite the feature’s being a more valid predictor of the US, such as when the target is more temporally contiguous with reinforcement than the feature is. Extending that reasoning, a feature might also acquire occasion setting when the target that was more associable with the US for nontemporal reasons. For example, even with simultaneous compounds in FP discriminations, if the target cue was considerably more intense than the feature, strong target-reinforcer associations would form, and overshadow conditioning to the weaker feature, despite the equivalent reinforcer contiguity of the feature and target and the superior predictive relation of the feature. The feature might then come to modulate the action of the already-established target-reinforcer unit, just as the more temporally-remote feature does in serial FP discriminations.

Consistent with this perspective, Holland (1989c; Holland & Haas, 1993), using Pavlovian or operant appetitive conditioning procedures (respectively) found that when a weak-moderate intensity auditory cue was used as the target with a visual feature, the rats solved the discrimination by acquiring feature-reinforcer associations, as in our previous simultaneous FP discrimination experiments, whereas when the target was a high-intensity auditory cue, the rats adopted an occasion setting strategy, using the light to set the occasion for responding to the auditory cue.

Target training.

If occasion setting depends on the formation of a target-US unit (which is in turn modulated by the feature), then training of the target-US relation prior to FP discrimination training should enhance the acquisition of occasion setting by providing a “head start” on the formation of the necessary target-US unit. Several studies examined the effects of target pretraining. Rescorla (1986b) found substantial enhancement of a feature’s occasion setting power in FP training in pigeon autoshaping, and Ross (1983) found a small enhancement in appetitive Pavlovian serial FP training in rats. Perhaps most interesting were the results of Holland (1989c), who examined the effects of prior target-reinforcer training on the acquisition of occasion setting in simultaneous FP learning in rats (described in the preceding section). In that experiment, target-pretraining facilitated the acquisition of occasion setting under conditions that produced occasion setting without such pretraining, (high target intensity) but did not encourage use of an occasion setting strategy under conditions that otherwise did not produce occasion setting (low target intensity). Thus, in that experiment, prior formation of a target-US unit alone was insufficient to encourage occasion setting: the perceptual conditions also had to be adequate.

Relative validity of feature and target.

Within our preliminary hypotheses, features acquire occasion setting when they are better correlated with reinforcement than targets that are more contiguous, more salient, or otherwise more easily associated with reinforcement. Thus, serial compound training alone should not establish the feature as an occasion setter, nor should procedures in which the probability of reinforcement is the same after feature+target compound and target-alone trials. Occasion setting is not observed if only reinforced feature+target compound are given, without also presenting nonreinforced target trials (e.g. Davidson & Rescorla, 1986; Holland & Ross, 1981; but see Bonardi, 1992), or if both the feature+target compound and target-alone trials are reinforced (Davidson & Rescorla, 1986, with rats and Rescorla, 1985 with pigeons). Similarly, Holland (1986b) found no evidence of occasion setting when a serial feature-target compound was reinforced on half of its presentations and the target alone was reinforced on half of its presentations; this observation was extended in unpublished experiments (described in Holland, 1992) to conditions in which 25% or 75% of both types of trials reinforced. Apparently, perceptual discontinuities or differences in the rates of acquisition to target and feature alone are insufficient for the development of occasion setting: the feature must also provide more information than the target.

Feature training.

Manipulations that enhance conditioning of the feature in FP discriminations tend to diminish occasion setting. For example, in a previous section we noted that temporal variables often affected feature conditioning and occasion setting in opposite manners. Similarly, Ross (1983) and Rescorla (1986b) showed that prior feature-reinforcer pairings interfered with the acquisition of occasion setting to that feature. It might be argued that the acquisition of conditioning to the feature directly interferes with its ability to acquire occasion setting, for example, from the perceptual view, by encouraging the grouping of feature and reinforcer, at the expense of the feature → (target→US) grouping. Alternately, there may be some other inherent competitive relation between conditioning and occasion setting, such that a feature may have only so much signal value to distribute among potential signaling functions.

A simpler account for these effects however is that prior training of the feature merely blocks the formation of target-reinforcer associations, which are demanded by each of the characterizations of the conditions necessary for the acquisition of occasion setting. This view is supported by Rescorla (1986b) who found that feature pretraining had deleterious effects only if that pretraining resulted in blocking of target-reinforcer associations. When blocking effects were minimized by explicit pretraining of the target as well, the feature readily acquired occasion setting properties. In fact, as noted when we discussed the effects of “Extinction and counterconditioning”, under some circumstances feature-reinforcer pairings can enhance the establishment of occasion setting, by increasing attentional processing of the feature. Thus, there seems to be little justification to assume any inherent competition between occasion setting and conditioning powers of a cue.

Feature nonreinforcement.

When we discussed the effects of “Extinction and counterconditioning”, we noted that converting a serial FP discrimination to a serial PP discrimination by adding contemporaneous nonreinforced feature presentations has little effect on the acquisition of occasion setting (Holland, 1989a, 1989b; Ross & Holland, 1981, 1982). Notably, this manipulation weakens both feature-US and feature-target associations and can make the feature no more valid a predictor of the US than the target. Thus, neither a consistent feature-reinforcer or feature-target relation, nor greater feature than target validity as a predictor of reinforcement, seems critical, as long as the feature remains a valid signal that a target, if presented, will be reinforced. Of course, our experiments involved only minimal degrading of those relations, ranging from 25% to 100% of all features being followed by targets, so it remains to be seen whether the acquisition of occasion setting is wholly immune to these effects. But suffice to say that within a range in which there was considerable variation in the amount of feature conditioning, there was no observable variation in occasion setting, just as was the case with variations in feature-reinforcer intervals, discussed in the section “Temporal factors”.

Spatial contiguity.

In the section “Temporal factors”, we showed that occasion setting is more likely to emerge when the feature and target are noncontiguous and discontinuous, such that the feature provides information whether reinforcement will occur and the target informs when it will occur. An analogous “whether-where” distinction may be made when cues are spatially separated, for example in landmark tasks, in which a feature in one location signifies whether reinforcement (or escape) is available, but responding must be directed toward one or more spatially noncontiguous locations (Leising, et al., 2015; Ruprecht, Wolf, Quintana, & Leising, 2014).

Context.

Perhaps the most significant “perceptual discontinuity” evident in typical conditioning experiments is that between the punctate cues typically used as CSs and the contextual cues in which those CSs are embedded. Many researchers, using a wide range of conditioning paradigms (e.g., Balaz, Capra, Hartl, & Miller, 1981; Bouton, 1984; Bouton & King, 1986; Bouton & Swartzentruber, 1986; Dibbets, Maes, Boermans, & Vossen, 2001; Goddard, 2001; Goddard & McDowell, 2001; Gonzalez, Garcia-Burgos, & Hall, 2012; Grahame, Hallam, Geier, & Miller, 1990; Holland & Bouton, 1999; Loy & Lopez, 1999; Lubow & Gewirtz, 1995; Mehiel, McCarthy, & Zellner, 1991; Murphy & Skinner, 2005; Swartzentruber 1991; Swartzentruber & Bouton, 1988) have suggested that contextual cues often act in ways that are reminiscent of the action of occasion setters. For example, analogs of the counterconditioning/ extinction experiments described in a previous section show the ability of contextual cues to modulate responding to discrete CSs to be distinct from the contexts’ simple excitatory or inhibitory relations with the reinforcer. Similarly, when the contributions of contexts’ simple associative relations with the reinforcer are eliminated, transfer of contextual cues’ modulating power is limited in ways similar to the limitations on transfer after occasion setting with explicit occasion setting cues (see the preceding and following sections on “Transfer effects”). Furthermore, Swartzentruber (1991) found that context cues and punctate cues trained explicitly as occasion setters provided redundant information in a blocking design, but context cues and simple punctate CSs did not, suggesting that contextual cues often act more like occasion setters than like simple CSs (see also Holland & Bouton, 1999). Nevertheless, there also are clear illustrations of contexts acting as simple CSs (e.g., Fukumoto, Sawa, & Ishii, 2014; Holland & Bouton, 1999; Iguchi, Ishii, Iguchi, & Sawa, 2006; Loy, Alvarez, Rey, & Lopez, 1993; Maes & Vossen, 1994.)

It is worth noting that in Swartzentruber’s (1991) experiments, contextual control of responding to a CS was established by explicit discrimination between the consequences of that CS in different contexts, analogous to the explicit discrimination procedures used in occasion setting experiments with discrete cues. On the other hand, other experiments that have suggested relations between contextual control and occasion setting (e.g. Bouton & Swartzentruber, 1986; Swartzentruber & Bouton, 1988) have used nondifferential procedures, which we did not find to be effective in establishing occasion setting when discrete cues are used as stimulus elements. However, as noted in the section on “Relative validity”, Bonardi (1992) found occasion setting with nondiscriminative training with discrete cues.

A number of investigators have posited important roles for occasion setting in extinction and related phenomena such as renewal and spontaneous recovery (Brooks, 2000; Delamater, 2012; Delamater & Westbrook, 2014; Fonteyne & Baeyens, 2011; Trask, Thrailkill, & Bouton, 2017), suggesting that in Pavlovian conditioning experiments, the contexts of reinforcement and of nonreinforcement act as positive and negative occasion setters for responding to explicit cues, or that explicit occasion setters may activate context-specific inhibitory CS-US associations (e.g., Bouton & Nelson, 1994; Nelson & Bouton, 1997). A growing consensus, however, is that free operant responding may be more directly controlled by simple inhibitory associations with the extinction context (e.g., Ahrens, Singer, Fitzpatrick, Morrow, & Robinson, 2016; Todd, 2013; Trask & Bouton, 2014). Given that discrete cues trained as occasion setters often possess simple excitatory or inhibitory powers as well, and that procedural variations may determine whether a discrete cue acquires occasion setting or simple associations, it should not be surprising that contexts also may act as simple CSs, occasion setters, or both.

Unfortunately, it is often difficult to specify what constitutes a context or a contextual cue, especially when those cues are somewhat hidden from view. For example, many investigators have suggested that, like external contextual cues, diffuse internal stimuli such as emotional or motivational states (e.g., Davidson, 1998, 2000; Holland, 1983) or drug states (e.g., Maes & Vossen, 1997, Maes, VanRijn, & Vossen, 1996; Palmatier & Bevins, 2008, Reichel, Wilkinson, & Bevins, 2007; Skinner, Martin, Harley, Kolb, Pridgar, Bechara, & van der Kooy, 1994; Skinner, Goddard, & Holland, 1998; Stolerman & Mariathasan, 2003; Troisi, LeMay, & Jarbe, 2010; Wilkinson, Li, & Bevins, 2009; see also “Transfer between two analogous occasion setting discriminations” section), may also be more prone to serve as occasion setters than as simple elicitors. This occasion setting tendency may hold true for more specific internal or hidden states as well, such as time-of-day cues (e.g., Means, Arolfo, Ginn, Pence, & Watson, 2000; Menzel, Geiger, Muller, Joerges, & Chittka, 1998; but see Delamater, Derman, & Harris, 2017) or recent reinforcement or trial history, as implied by hypothesized “contexts of reinforcement (or nonreinforcement)”, or the use of previous trial outcomes to cue alternation or successive reversal behavior (e.g. Wilson, Takashi, Schoenbaum, & Niv, 2014).

Given the differences between simple conditioning and occasion setting outcomes in many circumstances, consideration of contextual cues as occasion setters has important translational implications. Many investigators have posited occasion setting accounts of various psychopathologies, such as contextual reinstatement and relapse of drug seeking (Crombag, Bossert, Koya, & Shaham, 2008; Crombag, Grimm, & Shaham, 2002), sensitization from cocaine withdrawal (Gordon & Rosen, 1999), alcohol tolerance (Ramos, Siegel, & Bueno, 2002), and stimulant-induced psychomotor sensitization (Anagnostaras & Robinson, 1996; Agnagnostaras, Schallert, & Robinson, 2002), with predictions that differ from those generated by simple conditioning theories.

Common principles in the establishment of occasion setting

The data reviewed here suggest that occasion setting is most commonly observed under temporal or nontemporal conditions that encourage perceptual separation of feature and target, and/or favor more rapid formation of target-US than feature-US associations. Likewise, although not essential for its establishment, occasion setting frequently occurs when greater reinforcement predictive validity of the feature contrasts with greater salience or temporal/spatial reinforcement contiguity of the target. By contrast, Rescorla (1986b) downplayed the role of any special perceptual, temporal or predictive relations, but instead suggested that occasion setting occurs whenever the target acquires conditioning faster than the feature. Rescorla (1988) proposed a mechanism behind this rule: occasion setting is acquired when an occasion setter (or its trace) is reinforced in the presence of a target with a strong inhibitory component. In FP discriminations, the stronger the excitation initially conditioned to the target, the stronger the inhibition that accrues to that target when it is extinguished on target-alone trials, and hence the stronger the occasion setting that can accrue to the feature when it is reinforced on feature+target trials. Notably, many (but not all) of the conditions we described as encouraging occasion setting also arrange for the reinforcement of the feature in the presence of a target with a strong inhibitory component. Thus, the precise determinants of occasion setting are not completely specified.

Content of learning in occasion setting

Perhaps the most compelling question about any learning process concerns its content: How is the organism changed when it acquires occasion setting? In this section, we first consider the nature of the internal representation of occasion setting relations within a common framework for simple representations of knowledge in Pavlovian conditioning (e.g., Rescorla, 1974). In this framework, internal representations of CS and US events are linked by excitatory and inhibitory associations, which permit a CS to generate or inhibit CRs by activating or suppressing the US representation (e.g., Figures 1a and 1c). Next, we consider implications of apparent constraints on the eligibility of various kinds of stimuli to serve as suitable targets of occasion setting. Finally, we take up briefly questions about the properties and functions of occasion setters.

Loci of action of occasion setters and simple CSs

The evidence that the simple conditioning and occasion setting properties of cues can be independent, discussed in several previous sections, is most easily understood if associative and occasion setting links act at different loci. Consider first the simple associative representations of learning in FP and FN discriminations (Figures 1a and 1c). In FP discriminations, excitatory feature-US associations allow the feature to produce CRs by activating the US representation, and in FN discriminations, inhibitory feature-US associations allow the feature to inhibit responding by suppressing activation of the US representation by other CSs. Ross and Holland (1981) suggested that whereas simple excitation and inhibition act directly on the US representation, as just noted, occasion setting acts on the target-US association (Figures 1b and 1d). This locus of action would be consistent with the independence of occasion setting and simple conditioning: a single feature could easily have an inhibitory relation with the target-US association and an excitatory relation with the US representation, or vice-versa (e.g., dotted arrow in Figure 1d).

Another possibility (Figure 2a) is that the feature might modulate responding to the target by altering the ability of the target itself to activate its internal representation, for example, by directing attention toward or away from the target. Again, it is not difficult to imagine that a feature might simultaneously direct attention away from a target but have an excitatory association with the US. Conversely, Rescorla (1985) suggested that both simple CSs and occasion setters act directly on the US representation, but by different means. Whereas CSs elicit CRs by activating the US representation, occasion setters are linked to the US representation with a separate modulatory link, which transiently raises or lowers the US representation’s sensitivity to activation by its associates (Figure 2b). Thus a single feature might have at the same time both an excitatory association and an inhibitory modulatory link with the US representation.

Figure 2. Alternative associative structures established in serial feature negative discriminations.

Panel A shows a structure in which a representation of the feature (F) suppresses processing of a representation of the target (T), and Panel B shows a structure in which a representation of F suppresses processing or raises the activation threshold of a representation of the unconditioned stimulus (US). X refers to a representation of a transfer test stimulus paired with the US outside of the discrimination task (see sections on transfer effects in “Assays of occasion setting” and “Contents of Occasion setting”). Arrows indicate simple excitatory associations and open circles indicate negative modulatory (occasion setting) links. The dotted arrows refer to excitatory associations established after counterconditioning (see sections on counterconditioning in “Assays of occasion setting” and “Contents of Occasion setting”).

CS- and US-specificity of occasion setting.

A key tool in localizing the action of occasion setters is the transfer test. Early transfer experiments investigated the CS-specificity of occasion setting. If occasion setters acted by modulating the activity of the US representation, as Rescorla suggested, then they should alter responding to any cue associated with that US. However, Holland (see “Transfer effects” section) found that features trained within serial FP and FN discriminations failed to modulate responding conditioned to separately-trained target cues, leading to the conclusion that occasion setters did not act directly on the US representation, but on either the CS representation or on the CS-US association.

If occasion setters acted on the CS-US association, then their action should be US-specific as well as CS-specific. Thus, if the original target cue was paired with a new US after the completion of occasion setting training, then the feature should have no power to modulate responses based on the target’s associations with the new US. However, if the feature modulated attention to the target, then it should would alter the likelihood of the target’s eliciting any response conditioned to it, regardless of the US on which that CR is based. Experiments using appetitive FN discrimination procedures (Holland, 1985, 1989d), and which examined transfer after retraining with either other appetitive USs or aversive USs, showed that although the feature retained its ability to modulate any remaining responding that was based on the target’s association with the original US, it had no effect on responding due to the target’s associations with the new US. This US-specificity implied that the feature acted on the CS-US association itself, rather than on the target or US representations.

Consequently, Holland (1983, 1985) described occasion setting in terms of a simple hierarchical model, originally offered by Estes (1969, 1972) to account for a number of inhibitory effects in learning. In this model (similar to that in Figure 3) associations are represented as control elements that link representations of the individual events. Thus, in FP or FN discriminations, the connection between target and US representations is mediated by a target-US control element. In occasion setting, a link between the feature and the target-US control element (CE) is established (Figure 3); the feature modulates responding controlled by the target by facilitating or suppressing the activity of that control element. Holland suggested that procedures that establish occasion setting especially encourage the formation of these hierarchically organized representations of conditioning episodes, consistent with the perceptual discontinuity view noted early in the section on “Conditions for establishing occasion setting”. Other procedures instead promote the formation of associations between representations of the individual events themselves in the absence of any higher-level control elements. Holland (1983, 1985) then speculated that occasion setting might be a link between simple Pavlovian association and more complex learning, and that it might provide a model system for the study of hierarchical control.

Figure 3. Associative structure in multiple feature training.

Representation of associative structure after multiple feature positive (FP) or feature negative (FN) discriminations (after Estes, 1969). US1/US2 refer to two different unconditioned stimuli, and X and Y refer to stimuli separately paired with US1 or US2, respectively. F1/F2 and T1/T2 refer to the features and targets (respectively) used in two separate FP or FN discriminations. The circles labeled CE refer to control elements established as a result of this training, which modulate the associations between T1 and US1 and between T2 and US1 or US2 (dotted lines). See the section “Loci of action of occasion setters and simple CSs” for more explanation.

Transfer between two analogous occasion setting discriminations.

This particular hierarchical model demands substantial CS- and US-specificity of occasion setting, consistent with all of the transfer test data described so far. However, in several experiments in which two comparable occasion setting discriminations (e.g., two serial FP discriminations) were trained in each subject contemporaneously (e.g., Holland 1989a, d; Lamarre & Holland, 1987, Morell & Holland, 1993; Rescorla, 1985), substantial transfer of an occasion setter’s power to the target of another occasion setter was observed, despite no transfer to a third target that was trained separately, outside of any occasion setting discrimination. For example, Holland (1989d, Exp. 3) presented rats with reinforced trials with a simple CS, X+, intermixed with training on two serial FN discriminations: F1→T1−, T1→US1, F2→T2−, T2→US1, X→US1, using a food-pellet or sucrose US1. Subsequent transfer tests showed that each of the F1 and F2 features modulated responding to either T1 or T2, but neither affected responding to X. In addition, Morell and Holland (1993) found summation of the negative occasion setting powers of 2 features trained with different targets: a compound of the 2 features suppressed responding to either target more than each feature alone did. Notably, in all of these experiments, when other subjects were trained with only a single occasion setting discrimination, little or no transfer to other transfer targets was observed, across a range of treatments of those targets, including consistently reinforced or nonreinforced presentations, partial reinforcement, initial training and subsequent extinction, or training as the target of simultaneous FP/FN discriminations not solved using occasion setting strategies.

Although in Holland’s laboratory, these patterns of transfer (and failure of transfer) were broadly obtained (i.e., when either two FP or two FN discriminations were trained, and with either appetitive or aversive USs), Bonardi (e.g., Bonardi, 1998, 2007; Bonardi & Hall, 1994) has consistently reported less transfer of occasion setting with multiple discriminations than Holland. The reasons for this difference are unknown, but it does not appear to be simply the result of differences in generalization across feature or targets, or in ‘generalization decrement’ when novel compounds are presented in transfer tests (Bonardi, 1996; 1998; Holland, 1989a). Thus, in her descriptions of the content of occasion setting (considered later in this section), Bonardi has emphasized the specific coding of individual CS-US units, whereas Holland emphasized more general coding. More important, despite the differences in absolute amounts of transfer obtained, Bonardi’s, Holland’s, and Rescorla’s (e.g., 1991a, b) data fall on the same continuum: little or no transfer after training in a single occasion setting task, more (but seldom complete) transfer across targets of other occasion setters, and even after subjects are trained on multiple occasion setting tasks, little or no transfer to separately-trained targets (e.g., Bonardi, 1998; Bonardi & Hall, 1994). This general pattern is found in many conditioning preparations and species (e.g., Baeyens, et al., 2004; Cleland, Ruprecht, Lee, & Leising, 2017; Leising, et al., 2015; Roper, Chaponis, & Blaisdell, 2005; Skinner, et al., 1998).

Holland (1992; Lamarre & Holland, 1987) concluded that whereas training of a single occasion setting discrimination would establish modulatory links between the feature and the target-US unit (as in Figure 1b and 1d), training of multiple discriminations, with comparable treatment of multiple features and targets, might encourage a more global representation. That is, the higher-level control elements established in occasion setting training (Figure 3) may interact differently than do the event representations established with exposure to simple conditioning procedures. For example, the simple associative strength of a feature cue may be largely independent of the strength of its occasion setting powers, because the former involve simple links with individual event representations, whereas the latter involves independent links with control elements that relate those events. Likewise, transfer of simple excitation or inhibition occurs readily across cues of many training histories because all responding to cues is mediated by associations with a common US representation, but transfer of occasion setting occurs only to those cues that have themselves been involved in occasion setting, because only those cues are related by higher-level control elements.

To account for the observed transfer among targets of occasion setters in the structure portrayed in Figure 3, substantial generalization or class equivalence (Bonardi & Hall, 1994; Honey & Hall, 1989; Sidman, 1986) among either the features, the targets, or the higher-order control elements must be assumed. Holland (1992) favored equivalence among higher-order control elements, that is, for a feature F1 to exert influence over responding based on T2-US associations, the higher-order control elements that relate T1 and T2 to the US must be relatively interchangeable. Note that such enhanced generalization would apply only to control element interactions, and would not affect performance generated by interactions among simple event representations. Thus, a given target-US control element might be a suitable target for the action of other occasion setters, but lower-level representations of individual CSs and USs would not be (e.g., X’s relation to US1, or T1’s to US2 in Figure 3).

Bonardi (e.g., 1998, 2007) instead emphasized the coding of specific CS-US associations, regardless of the number of occasion setting tasks encountered, dispensing with the notion of a more global representation after more complex training. To account for greater transfer after training on multiple occasion setting tasks, Bonardi simply asserted that such training established like members of the tasks (that is, the features, the targets, or the reinforcers) as functionally equivalent, at least in part. Several experiments provided independent evidence for such enhanced generalization (e.g. Bonardi & Hall, 1994; Honey & Hall, 1989). With this addition, Bonardi’s and Holland’s representations of occasion setting become functionally very similar; both encode specific CS-US associations, but provide for greater transfer across occasion setting tasks by assuming acquired equivalence across either stimulus elements within those associations or superordinate control elements.

An implication of either of these structures is that occasion setting would transfer across USs if they were both trained within discriminations known to produce occasion setting, and hence establishing functional equivalence of the USs or control elements involving each of them (changing the T2-US1 control element to a T2-US2 control element –the dotted line in Figure 3-adapts the hierarchical framework in that figure to this situation). Holland (1989d, Experiment 4) first trained rats with two serial FN discriminations, with two different USs: T1→US1, F1→T1−, T2→US2, F2→T2−. After separate pairing of a third excitor with one of the USs, (X with US1 or Y with US2), a transfer test examined F1’s and F2’s abilities to suppress responding to T1, T2, and X or Y. Transfer of F1’s and F2’s negative occasion setting powers to the targets that had been paired with the other US was substantial, but neither feature affected responding elicited by X or Y, regardless of its associate (but see Bonardi, Bartle, Jennings, 2012, for an observation of US-specific transfer to a separately-trained transfer target). Thus, in Holland’s experiment, transfer of occasion setting across both CS targets and USs was observed, but only if the transfer targets and USs had themselves been part of an occasion setting discrimination, and hence control elements linking those events had been established. Accordingly, in this experiment responding mediated by any target-US control element seemed to be an appropriate locus for the action of occasion setters. By contrast, using a somewhat different, blocking, design, Bonardi (e.g., Bonardi, 1998, 2007; Bonardi & Ward-Robinson, 2001) found evidence for substantially more US-specificity of occasion setting (less transfer) across USs in both pigeon autoshaping and rat appetitive conditioning.

The framework shown in Figure 3 captures the intuitions that interactions among elements involved in occasion setting and those that aren’t may follow different rules and may be largely independent of each other, and that animals may apprehend and represent hierarchical relations among events. However, Holland (1992) noted that simply positing that representations of events that participate in occasion setting follow rules different from those that apply to events that participate only in simple associative relations does a reasonable (and occasionally, better) job describing differences in transfer and other characteristics of occasion setters and simple CSs. For example, Holland (1990b, 1992) suggested that representations of events involved in occasion setting are processed in a separate, higher-level (perhaps hippocampal) memory system than representation of events that participate only in simple associative relations, and that these memory systems interact only minimally. Of course, to say that higher-order target-reinforcer control elements need not be represented for transfer to occur does not demand that they cannot be represented. It is clear that after training on a single task, occasion setting is highly specific, and even with multiple task training, transfer is seldom complete. This specificity and lack of complete transfer suggests representation of specific target-reinforcer associations or control elements as well as of individual events and may form the basis of more hierarchically complex representations (e.g., Colwill & Rescorla, 1990).

Although a great deal of evidence supports this general statement, we note that the particulars of transfer may nonetheless vary depending on the conditioning preparation, species, and events used (Skinner, et al., 1998). For example, Goddard (Goddard, 1999; Goddard & Holland, 1996, 1997) found substantial transfer of occasion setting after training two serial FP (or FN) discriminations using 2 visual features or 2 flavored sucrose features, but no transfer if one feature was visual and the other was flavored sucrose. This observation indicates limits to the apparent equivalence observed after training with multiple occasion setting tasks. Interestingly, substantial transfer occurred if one feature was visual and the other a flavored but non-nutritive and unsweetened solution (Skinner, Thornton, & Holland, 2003), suggesting that the “biological significance” (Denniston, Miller, & Matute, 1996) of features may play a role in determining transfer. Likewise, in pigeon autoshaping Nakajima (1997b) found substantial transfer across occasion setting tasks with keylight features but not with diffuse features. Finally, investigations of occasion setting using drug states as occasion setters and/or targets have produced a variety of transfer patterns (e.g., Maes & Vossen, 1997, Maes, et al., 1996; Palmatier & Bevins, 2008, Reichel et al., 2007; Stolerman & Mariathasan, 2003; Troisi et al, 2010, Wilkinson, et al., 2009). For example, Skinner et al. (1994, 1998) found that transfer of occasion setting of a learned flavor aversions differed depending on whether the occasion setter was a drug state, a context, or another flavor.

Independence of positive and negative occasion setting.

Although the preceding discussion has emphasized privileged transfer among events involved in occasion setting, it seems unlikely that all events involved in occasion setting would be classed equivalently. For example, Holland (1992) noted that early experiments showed no evidence that features and targets were mutually replaceable, that is, events trained as features could not serve as targets, and vice-versa. And it seems unlikely (although we know of no relevant data) that the breadth of transfer of occasion setting across USs would extend to those from different motivational systems, e.g., those involving food and shock USs (Bonardi & Jennings, 2009).

Existing data make it clear that positive and negative occasion setting are often independent. The experiments described in the previous section examined the transfer of occasion setting across targets and USs that had been involved in 2 discriminations of the same “type”, either positive (as in FP discriminations) or negative (as in FN discriminations). In this section we examine the extent to which occasion setting transfers across type, that is, whether features from FP discriminations modulate responding elicited by cues trained as targets of FN discriminations, and vice-versa.

Javier Morell (described in Holland, 1992) first trained rats with both a FP and a FN discrimination (F1→T1+, T1−, T2+, F2→T2−) and later assessed F1’s and F2’s abilities to modulate responding to the other targets in a set of transfer tests. To facilitate the observation of both suppressive and facilitatory effects of the features on both targets, prior to one transfer test, Morell reinforced T1 and nonreinforced T2 until responding occurred to both at an intermediate level. In all of the transfer tests, although F1 still enhanced responding to T1 and F2 suppressed responding to T2, F1 had no effects on responding to T2 and F2 had no effects on responding to T1. Given that in previous experiments using similar training parameters (described in the preceding section), there was substantial transfer when both discriminations were of the same type, the absence of transfer in this experiment implied substantial independence of positive and negative occasion setting.

Furthermore, in this same experiment Morell also examined the effects of an F1+F2 compound on responding to T1 and T2. If F1’s and F2’s occasion setting powers summed (subtractively), then the compound should yield intermediate levels of responding to both T1 and T2, relative to the effects of F1 or F2 alone. But if only F1 can act on T1 and only F2 can act on T2, then the F1+F2 compound should facilitate responding to T1 equivalently to F1 alone, and suppress responding to T2 equivalently to F2 alone. Morell found the latter outcomes, further strengthening the idea that positive and negative occasion setting are quite independent (but see Rescorla, 1987, who found the former result in a similar experiment in pigeon autoshaping).

Holland (1991b) and Holland and Reeve (1991) further explored the relation between positive and negative occasion setting by examining learning of discriminations in which a feature served contemporaneously as a positive occasion setter for responding to one target and as a negative occasion setter for responding to another target (F→T1+, T1−, T2+, F→T2−). If positive and negative occasion setters summed or transferred, because, for example, they operated by raising or lowering (respectively) the sensitivity of a common US representation to activation by target cues (Rescorla, 1985, 1987), then this “ambiguous feature” discrimination should be very difficult because the feature would have to both raise and lower that threshold before any target was delivered. In fact, this discrimination was learned readily: acquisition of each component of the ambiguous discrimination (F→T1+, T1− and B+, F→T2−), was at least as rapid as learning in control rats that received only the FP or only the FN discrimination, suggesting that positive and negative occasion setting powers of the same feature were acquired independently. Within occasion setting structures like those in Figures 1b, 1d, or 3, this outcome is expected, because there would be no reason the feature could not both activate the T1-US association and suppress the T2-US association.

Holland (1991b) examined the nature of learning in these ambiguous discriminations by examining transfer among the features and targets of ambiguous, FP, and FN discriminations. All rats received training on an ambiguous discrimination and either another ambiguous discrimination, a FP, a FN, or a nondiscriminative control procedure. The transfer data showed that the ambiguous feature displayed transfer properties like those displayed by positive and negative occasion setters from FP and FN discriminations. Besides transferring readily to both targets of the other ambiguous discrimination, the ambiguous feature enhanced responding to the target of the FP discrimination, suppressed responding to the target of the FN discrimination, and had no effect on the control target (essentially, a “separately trained cue”). Furthermore, the feature trained within a FP discrimination enhanced responding to the positive (T1) target of the ambiguous discrimination, but had no effect on the negative T2 target of the ambiguous discrimination, and the feature trained within a FN discrimination suppressed responding to T2 but had no effect on responding to T1. Thus, positive and negative occasion setting functions in the FP and FN discriminations were independent (as in Morell’s experiment described above) and the two targets of the ambiguous discrimination were specific in their ability to be modulated by features trained solely as positive or negative occasion setters. Taken together with the acquisition data, these transfer effects suggest that positive and negative occasion setting are acquired and expressed independently, at least in this setting.

Other eligible targets of occasion setting

The preceding discussion can be summarized as suggesting that only stimuli trained as targets within occasion setting discriminations can be modulated by other occasion setters, and that such transfer is specific to occasion setting valence (positive or negative). Data from Holland’s laboratory have routinely supported the view that responding to stimuli trained outside of procedures that generate occasion setting are not modulated by occasion setters, a view that influences conceptualization of the content of learning in occasion setting as described in the preceding four sections. However, research from other laboratories indicates important exceptions to this claim. Thus it is important to consider the conditions under which CSs become appropriate targets for the action of occasion setters, just as we previously considered the conditions under which feature cues acquired occasion setting properties.

The data presented in the preceding sections indicate that it is sufficient that a cue be trained as a target of another occasion setter with similar-valence relation with the reinforcer. An important question is whether that training is necessary. Most data (e.g., Bouton & Swartzentruber, 1986; Holland, 1986b; Rescorla, 1985) are consistent in indicating that some treatments do not endow cues with the ability to serve as such a target (e.g., consistent or partial reinforcement). However, the evidence on at least two other kinds of target is mixed. Rescorla (1985) suggested that any cue that has both excitatory and inhibitory components, such as an FT compound trained in a simultaneous feature negative discrimination (T+, FT−), or a trained and extinguished cue, would serve as an appropriate target for modulation by positive occasion setters. Rescorla (1987) found such transfer in occasion setting with a pigeon autoshaping preparation, whereas Holland (1986b) found no evidence for transfer to such cues in a rat appetitive conditioning procedure. Likewise, although in our laboratory, responding to a trained and extinguished stimulus (which is widely believed to possess both excitatory and inhibitory associations with the US) is unaffected by feature presentation after serial Pavlovian appetitive occasion setting training (e.g. Holland, 1983; 1986b; 1989a, 1989b), other investigators (e.g., Davidson and Rescorla, 1986; Jarrard & Davidson, 1991; Rescorla, 1985; Swartzentruber & Rescorla, 1994) routinely found occasion setters (or context cues, Bouton & Swartzentruber, 1986) to enhance responding to such extinguished cues. Indeed, Holland (1991a) found reliable (but not complete) transfer to an extinguished cue in a discrete-trial operant lever pressing occasion setting procedure. No account for this discrepancy has been offered. But it is clear that to the extent that events such as those just described are acceptable targets of occasion setters, claims about the necessity of occasion setting training for establishing transfer-eligible control elements are suspect, unless one assumes that occasion setting training is implicit in these other training procedures. For example, Bouton and Swartzentruber (1986) argued that the operations of training and extinction establish a CS as a target of an occasion setter, which can be identified as the “context of reinforcement”.

Properties and functions of occasion setters

Thus far, we have discussed occasion setting in abstract terms such as modulation of a target-US association. However, all of our examples involved the modulation of the ability of a target CS to elicit a CR, one of many functions CSs acquire as a result of learning. For example, CSs can come to retrieve detailed sensory properties of the US (e.g., Holland, 1990a) and acquired incentive motivational properties, such as the ability to modulate instrumental (PIT; Holland, 2004; Holmes, Marchand, & Coutureau, 2010) or consummatory (Holland & Petrovich, 2005; Johnson, 2013) behavior, or serve as a conditioned reinforcer for the establishment of new learning (Everitt & Robbins, 2005). There have been few attempts to determine if occasion setters also modulate these other acquired properties of target CSs. Notably, a number of behavioral (e.g., Holland, 2004) and neural (e.g., Holland & Petrovich, 2005) manipulations show that these properties of simple CSs are multiply dissociable.

Modulation of sensory information.

Evidence from devaluation (Holland, 1990a), differential-outcome expectancy (McDannald, Saddoris, Gallagher, & Holland, 2005), and mediated learning (Holland, 1990a; Wheeler, Sherwood, & Holland, 2008) experiments shows that CSs often retrieve specific sensory information about their associated USs. For example, if after delivering pairings of 2 CSs with 2 foods that differ only in flavor (CS1→US1, CS2→US2), the subject’s evaluation of one of the USs (US1) is devalued (by selective satiation or pairing with an illness-inducing agent) in the absence of either CS, subsequent responding to CS1 is reduced relative to responding to CS2, as if CS1 were retrieving a specific representation of US1.

It is of interest to ask whether a target CS’s ability to retrieve a sensory representation of the US is modulated by an occasion setter. Observations of US specificity of occasion setting suggest that they can; at least after training with a single occasion setting task, features tend to only modulate CRs based on the target’s association with the original US, and Bonardi has observed US-specificity of multi-task occasion setting (see the section on “Transfer between two analogous occasion setting discriminations”). Likewise, Holland (1999) found that devaluing one US by pairing it with LiCl injection after training with multiple features, targets, USs, and responses in operant serial feature positive discrimination tasks selectively reduced responding based on that US in a variety of transfer tests. Furthermore, Delamater, Kranjec, and Fein (2010), using a differential outcome expectancy procedure, found that arranging consistent target-US relations enhanced occasion setting in ambiguous (FT1→US1, T1−, T2→US2, F→T2−) and biconditional (F1T1→US1, F1T2−, F2T2→US2, F2T1−) discrimination learning tasks compared to conditions in which either US1 or US2 was delivered on reinforced trials. And, Holland (1999) found that the rate of learning of two operant serial FP discriminations was affected by whether the target-US relations were consistent (F1T1(response)→US1, T1−, F2T2(response)→US2, T2−) or inconsistent (either US1 or US2 delivered on both types of reinforced trials).

Modulation of incentive motivation.

CS-food pairings endow the CS with the ability to serve as a reinforcer in subsequent Pavlovian second-order conditioning (CS1→US ∣ CS2→CS1) or instrumental secondary reinforcement (CS→US ∣ response→CS) procedures. Do occasion setters also modulate a target CS’s ability to serve this conditioned reinforcement function? For example, after a serial FP discrimination, would the target be an effective conditioned reinforcer if accompanied by the occasion setting feature, but not if presented alone? Fraser and Janak (2018) recently explored these issues, assessing the conditioned reinforcing value of features and targets, both alone and in combination, after rats learned a serial positive patterning discrimination (F→T+, F−, T−). In tests of conditioned reinforcement, rats worked avidly to earn the combination of the feature and the target. Interestingly, rats also worked to obtain the feature alone, but did not work for the target cue in isolation. This outcome contrasts with what would be expected from studies of simple Pavlovian CSs: although the feature and target were both reinforced with the same probability, the target was more contiguous with reinforcement, and thus should have been the more potent conditioned reinforcer. Nevertheless, to evaluate possible influences of simple feature-US (or second-order feature-target) conditioning, Fraser and Janak conducted conditioned reinforcement tests in a separate group of rats for which CRs to the feature were explicitly extinguished after training, and found that rats still worked to earn the feature in isolation as well as to earn the feature+target compound.

The observation that the feature’s ability to serve as a conditioned reinforcer by itself survived extinction may imply that this incentive function is acquired by occasion setters because they provide incentive information that is not affected by feature-alone presentations. Feature-alone presentations do not negate the possibility that the target, if presented, would be followed by the US. Within this view, the feature evokes an emotional and motivational state that prepares the animal to respond vigorously and efficiently if presented with the appropriate cue in the future. This notion dissociates the incentive predictive properties typically attributed to simple Pavlovian CSs to two distinct cues, with the incentive properties attributed to the feature and the predictive properties to the target. However, another possibility is that, unlike CRs, simple conditioning of incentive to the feature may survive nonreinforcement, in the same way that associations of CSs with sensory features of USs survive extinction, as assessed in tests of Pavlovian-instrumental transfer (Rescorla, 1993, 1996). Although evidence from second-order conditioning studies (e.g., Holland, 2016; Lindgren, Gallagher, & Holland, 2003) suggests that it does not, it would be of interest to determine if the conditioned reinforcement powers of a simple CS (that is, one not also trained as an occasion setter) survives extinction in Fraser and Janak’s setting.

Theories of occasion setting

Theories of occasion setting run the gamut from casual descriptions to formal models. Likewise, some emphasize the similarity of occasion setting and other associative processes and others stress the uniqueness of occasion setting, espousing a special modulatory process distinct from simple association, as we have done. These theories have been reviewed elsewhere (e.g., Bonardi, et al., 2017; Delamater, 2012; Schmajuk, et al., 1998); here we mention only a few, which we think exemplify approaches that should be kept in mind as we seek neurobiological correlates and mechanisms.

Hierarchical models

This review, and much early work on occasion setting, was based on a casual notion of occasion setting as a special modulatory function that acted on representations of the CS, US, or CS-US association to gate or otherwise modify the ability of CSs to elicit CRs, presumably by activating representation of USs. Because most of the evidence favored modulation at the level of CS-US associations, the idea of occasion setting as a model of hierarchical organization was popular, if poorly specified (Bonardi et al., 2017; Holland, 1983, 1985; 1992). Because many views of brain function emphasized hierarchical organization within brain and memory systems, the idea of occasion setters’ being represented in some higher-level system (e.g., the hippocampal formation) acting on control elements or associative units constructed at another level was appealing (e.g., Holland, 1992; Holland & Bouton, 1999; Myers & Gluck, 1994; Schmajuk & Buhusi, 1997), and seemingly supported by early data showing hippocampal dependence of some aspects of occasion setting, but not simple CR elicitation and inhibition (e.g., Holland, Lamoureux, Han, & Gallagher, 1999; Ross, Orr, Berger, & Holland, 1984). Unfortunately, although these approaches proved useful in organizing data and inspiring data collection, they have not been developed formally, except in the context of network models (considered in a subsequent section).

Elemental modulatory theories

Although throughout this review we have emphasized distinctions between occasion setting and simple association, it is clear they also have much in common. Most well-studied conditioning phenomena have parallels in occasion setting, for example, overshadowing (Gunther, Cole, & Miller, 1998), blocking (Bonardi, 1991; Bonardi & Hall, 1994; Bonardi & Ward-Robinson, 2001; Swartzentruber, 1991), higher-order conditioning (Arnold, Grahame, & Miller, 1991), latent inhibition, and learned irrelevance (Oberling, Gunther, & Miller, 1999).

The idea of modulatory as well as response-eliciting consequences of conditioning is not unique to occasion setting. For example, response elicitation and incentive motivational properties of simple CSs are often distinguished, as in Pavlovian-instrumental transfer (Holmes, et al., 2010), in which Pavlovian CSs paired with food can enhance instrumental responding reinforced by food, in both food-specific and general manners. As with occasion setting, this modulatory power of CSs is often independent of those CSs’ ability to elicit CRs (e.g., Holland, 2004), and depends somewhat on the training history of their response targets (Holmes, et al., 2010). Likewise, a variety of brain manipulations are known to abolish Pavlovian–instrumental transfer while leaving response elicitation intact (Holmes, et al., 2010).

These conditioned emotional responses might also modulate the acquisition and performance of other Pavlovian responses. For example, Konorski (1967) proposed that CS-US pairings produced separate associations between CS representations and two kinds of US representations, one mostly affective and one mostly sensory-motor. Activation of the affective representation not only generated certain kinds of CRs, but also modulated the performance of CRs mediated by the sensory-motor representation. In the context of a related formal model (AESOP, e.g., Wagner & Brandon, 1989), which assumes similar mechanisms for the acquisition of excitation and inhibition for both affective and sensory-motor association learning, but different parametric characteristics, Brandon and Wagner (1991) suggested that some occasion setting phenomena may reflect this PIT-like sort of modulation. However, as they (and Holland, 1992) noted, this approach was unable to account for many aspects of transfer and of the conditions required for the acquisition and extinction of occasion setting, without additional assumptions of different characteristics for emotional and sensory-motor associations (such as dramatically different susceptibility to extinction and counterconditioning). Nevertheless, this general approach proved useful in later elaborations, described in the subsequent section on “Added and replaced elements theories” (see also Nakajima, 1997a).

Configural theories

An alternative account for many of the phenomena discussed here eschews special modulatory processes like occasion setting, and emphasizes instead the conditioning of compound stimulus configurations (e.g., Brandon & Wagner, 1998; Pearce, 1987, 1994; Soto, et al., 2014; Wagner & Brandon, 2001; Wilson & Pearce, 1989, 1990; Young & Pearce, 1984). Within such theories, a compound cue AX is not treated as the simple combination of elements A and X, but as something more or less than the sum of its parts, or, in the extreme, as a unique stimulus.

Unique stimulus accounts.

We first consider unique stimulus accounts, in which stimuli that elemental theories describe as compounds and their elements are considered as independent stimuli, which may generalize to each other. For example, in a FP discrimination procedure, the feature+target acquires excitation, which generalizes to the feature, the target, and any transfer stimuli, according to specified rules of generalization. The ease of solving the FP discrimination depends on the ability to distinguish the unique feature+target configuration from the target alone. Because these stimuli are independent, it would be simple for responding to the feature+target to be relatively independent of responding to the feature or target stimuli, and for transfer to other targets to be limited, both hallmarks of occasion setting. However, because it seems reasonable to assume that the ease of forming such unique configurations should be based on perceptual principles, such as Gestalt grouping by similarity, contiguity, continuity and so forth, some researchers (e.g., Bonardi, et al., 2017; Holland & Lamarre, 1987) rejected a configural approach to occasion setting: occasion setting seems to be discouraged rather than encouraged by feature-target similarity, contiguity, and continuity (as described in the section on “Conditions for establishing occasion setting”). Notably, unlike occasion setting when serial compounds are used, configural learning when simultaneous compounds are used does seem to be facilitated under these conditions, consistent with generalization theories that relate the amount of generalization to perceptual stimulus dimensions (e.g., Harris, 2006; Kinder & Lachnit, 2003; McLaren & Mackintosh, 2001; Soto, et al., 2014).

This apparent ‘perceptual’ shortcoming of configural models derives from the assumption that these perceptual variables determine the ease of constructing distinguishable configural stimuli, which is not the case with all such models. In this section we discuss in some detail a theory of generalization proposed by Pearce (1987, 1994) to show the power (and some shortcomings) of alternative configural approaches. In this, as in other configural models, in a FP discrimination procedure, the feature+target acquires excitation, which generalizes to the feature, the target, and any other (for example) transfer stimuli, according to specified rules of generalization. However, within Pearce’s theory it is not a question of the subject’s being more or less likely to configure a particular compound: compounds are always configured. Variables such as the similarity and temporal relations between feature and target instead affect the amount of generalization between the compound configuration and the stimuli other theorists would refer to as its elements. Coupled with simple assumptions about the rules of generalization (in their simplest form, the more salient the target, the more the feature+target will resemble the target than the feature), this theory can account for many (but not all, Holland, 1992) of the phenomena described in this article. In this section we describe how Pearce’s configural model of conditioning accounts for the three basic findings we used to distinguish occasion setting from simple conditioning: response form, counterconditioning/ extinction, and transfer effects (see “Assays of occasion setting”).

In this model, differences in response form observed with serial and simultaneous FP discriminations reflect differences in generalization between the ‘compound’ and individual stimuli. In a serial FP discrimination, the feature→target compound at the time of reinforcement might be redescribed as a serial combination of the feature and a simultaneous compound that comprises both the target and the feature’s trace. Because the target is likely to be more salient than the feature’s trace, this compound is likely to resemble the target more than the feature. Thus, the form of responding observed during this portion of the serial compound (i.e., during what we describe as the target) is likely to be characteristic of responding generated by cues with the target’s physical characteristics, and will generalize substantially to the target alone and less to the feature alone. Consequently, the serial discrimination would be relatively difficult (it usually is, e.g., Ross & Holland, 1981) and relatively lower levels of simple conditioning to the feature might be anticipated.

In a simultaneous FP discrimination, the form of responding during the compound would also be determined by the physical characteristics of that compound. If that compound resembles the feature more than the target (i.e., if the feature is more salient), then the form of the response conditioned to the compound should be more like that observed to the feature. Furthermore, that strength should generalize substantially to presentations of the feature alone. Conversely, relatively little strength should generalize to the target, so the simultaneous FP discrimination should be easier than a serial discrimination with the identical elements (as Ross & Holland, 1981, noted). However, if the target were more salient than the feature, then the pattern of responding would mimic that described for the serial FP task, consistent with what Holland (1989b) observed when very salient target cues were used.

Thus, this configural theory captures the major aspects of the differences in response form observed in various FP discriminations, without recourse to a modulatory process. Furthermore, it does so without having to assume that configuring is more likely with serial than with simultaneous compounds. Temporal variables have their effects, not by encouraging a configural process, but by modifying the similarity of the compound to the feature and target elements: arranging the cues serially, and placing gaps between them (both of which encourage the pattern of data we describe as occasion setting), reduce the relative salience of the feature, enhancing the generalization between compound and target.

Within Pearce’s theory, compounds are unique cues that acquire excitatory and/or inhibitory strengths independent (except for generalization) of those of their so-called elements. Thus, it is hardly surprising that under many circumstances, post-training manipulation of the associative strength of the feature alone (extinction or counterconditioning) may have relatively little effect on responding to the compound. For example, in FP discriminations, if the feature+target cue resembled the target more than the feature (as with serial FP discriminations, or if the target was more salient than the feature) then the feature would possess relatively little generalized excitatory strength. Consequently, nonreinforced presentations of the feature would have little opportunity to produce inhibitory learning that could generalize to the feature+target, and so would have little effect on responding to that compound. Conversely, in simultaneous FP discriminations with less salient target cues, the compound would generalize substantially to the feature, which in subsequent extinction would accrue substantial inhibitory tendencies, in turn generalizing substantially to the compound, reducing its net strength. All of these predictions are supported in the data (e.g. Holland, 1989b; Young & Pearce, 1984). Analogous predictions can be derived for the case of excitatory counterconditioning of the features after FN discrimination training.

Finally, given the simple assumption that the target portion of a serial compound resembles the target alone more than the feature alone, Pearce’s (1987) theory can account for the basic differences in transfer observed with serial and simultaneous FP and FN discriminations. The more the compound resembles the target, the more generalization decrement will occur when the training target is replaced with a test target, because the test compound will be very different from the training compound (casually speaking, because they share only a weak feature). Thus, there will be less transfer after serial FP and FN discrimination training than after simultaneous training, in which the training and test compound generalize substantially (casually, because they share a salient feature). Furthermore, transfer might be greater among targets of multiple discriminations because there might be more sources of generalization for the test compounds.

Added and replaced-elements theories.

Some accounts for occasion setting involve a blend of elemental and configural processes. For example, Wagner and Rescorla (1972) suggested that a compound AB might comprise A, B and an additional cue unique to the compound, all of which compete on equal footing for excitatory and inhibitory learning. In a FP discrimination, apportionment of learning to the feature, target and unique feature+target stimuli would depend on the relative salience of the unique and elemental cues. To the extent that the unique cue acquired conditioning, specificity of responding to the original compound and resistance of compound responding to feature extinction would be anticipated. However, as noted in the previous section, to accommodate most occasion setting data, these theories must assume that the unique cue would be more salient when the elements were separated in time and perceptually dissimilar, which seems counterintuitive.

Wagner (e.g., Brandon & Wagner, 1998, 2001; Vogel, Ponce, & Wagner, 2017; Wagner, 2003) suggested a more complex way of representing the interaction of stimuli. Within this “replaced elements” scheme, in addition to a set of “context-independent” elements that are activated whenever a stimulus is presented, there are context-dependent elements that replace each other in the presence or absence of another stimulus. So, in a FP discrimination, when the feature is presented in compound with the target, it is represented as F_i+F_t, but when it is presented alone, it is represented as F_i+F_no-t. By logic essentially similar to that just described for Pearce’s (1987) theory, within this theory, control by configural (that is, context-dependent) elements would be greater with serial than with simultaneous compounds. With serial compounds, the context-independent feature elements (F_i) are in a poorer temporal position to compete for association on reinforced compound trials, relative to the target (context)-dependent F_t elements, so the bulk of the learning about the feature accrues to the configural (context-dependent) F_t elements. Hence, subsequent presentations of the feature alone (which does not activate F_t elements) would have little effect on responding to the compound, and presentation of F alone or in compound with other targets would not yield transfer responding, again, because the conditioned F_t elements would be absent. By contrast, with simultaneous compounds, F_i elements would be more competitive in acquiring associative strength and hence presentation of the feature alone or with other targets would be expected to elicit CRs. Importantly, Vogel et al. (2017) provided a real-time computational model, which predicted most of the effects of variation in temporal and similarity relations, the contrast between transfer after training with single and multiple discriminations, and the independence of positive and negative occasion setting described in this review.

Utility of configural models.

In summary, a well-specified configural view can provide a useful counterpoint to views such as ours, which propose a special stimulus control function of occasion setting. Many of the basic aspects of the data we have described as favoring an occasion setting view are compatible with such configural views as well, although many investigators have pointed out exceptions (e.g., Bonardi et al., 2017, Delamater, Garr, Lawrence, & Whitlow, 2017, Holland, 1992, Skinner et al., 1994). Thus, it is important to recognize that the mere passing of two or three test assays does not guarantee that a modulatory process has been demonstrated.

Neural network models of occasion setting

A variety of network models positing ‘hidden units’ to accomplish configuration-like processes have been applied to occasion setting (e.g., Delamater, 2012; Kehoe, 1988; Myers & Gluck, 1994). Here, we describe a model of occasion setting offered by Schmajuk and colleagues (e.g., Schmajuk & Buhusi, 1997; Schmajuk, et al., 1998), which combined features from both hierarchical and configural models. This model shared with most theories of associative learning the idea that learning occurs when there are discrepancies between USs actually received and the aggregate prediction of those USs based on excitatory and inhibitory associations of CSs (e.g., Rescorla & Wagner, 1972). Unlike the Rescorla-Wagner model (1972), it was a “real-time” (e.g., Sutton & Barto, 1981) model, and it incorporated a layer of hidden units, which could code configural stimuli, between input CSs and output US/CR units. CSs were assumed to be coupled to output units both directly and indirectly, through the hidden units. Thus, in this model, a cue acts as a simple CS through its direct associations with output units and as an occasion setter thorough its indirect connections via the hidden units. An important feature of the model was that the initial connection weights of the five hidden units varied considerably, such that different units might carry the brunt of the associations depending on task demands.

The model was able to simulate most of the results described in this review, including the differences in response form and transfer observed between simultaneous and serial FP and FN discriminations, as well as the independence of the simple conditioning and occasion setting powers of serially-trained features, and the effects of variations in temporal and similarity relations previously described. Interestingly, the model’s solutions of the various tasks that yield occasion setting all involved the hidden units coming to inhibit output units, in both positive and negative occasion setting. Thus, in a serial FP discrimination, the target acquired direct excitatory associations with both the hidden units and the output units, whereas the feature came to inhibit the hidden units (and to have only a negligible effect on the output units). When the target was presented alone, the effect of the target’s excitatory associations with the output units was cancelled by its activation of the hidden units, and hence the suppression of the output units. But when both feature and target were presented on a trial, the feature inhibited activity of the hidden units, releasing their normal inhibition of the output units, and permitting output unit activation by the target. Likewise, in a serial FN discrimination, the target acquired strong direct excitatory associations with the output units, and weaker ones with the hidden units, but unlike in the FP discrimination, the feature acquired excitatory associations with the hidden units. Thus, on target-alone trials, the output units were strongly activated by the target, whereas on feature+target trials the inhibitory hidden units were strongly activated by the feature, counteracting the target’s effects. Interestingly, these solutions share much with a proposal by Nelson and Bouton (1997) that occasion setting involves the modulation of the effectiveness of inhibitory target-US associations, with positive occasion setters suppressing, and negative occasion setters enhancing, their action.

Implications of occasion setting for theories of associative learning.

Certain training procedures appear to endow stimuli with the ability to modulate the action of other stimuli. Although there is little consensus on a definitive model, this occasion setting function seems conceptually and empirically distinguishable from simple conditioning in many ways. Indeed, under many circumstances the occasion setting and simple conditioning powers of a single cue may be quite independent.

Occasion setting has been reported in a broad array of species, for example, humans (Baeyens et al., 2004), rats (Holland, 1992), mice (Shobe, Bakhurin, Claar, & Masmanida, 2017), pigeons (Rescorla, 1985), honeybees (Giurfa & Menzel, 2003; Mota, Giurfa, & Sandoz, 2011), cockroaches (Matsumoto, Matsumoto, Watanabe, Nishino, & Mizunami, 2012), flies, (Brembs & Weiner, 2006), and the nematode c. elegans (Law, Nuttley, & van der Kooy, 2004). It has been implicated in many conditioning phenomena that do not involve explicit occasion setting procedures, including the development of contextual control of behavior (“Context” section), renewal, reinstatement, relapse and spontaneous recovery after extinction (“Context” section), latent inhibition (Lubow & Gewirtz, 1995), conditioned inhibition (Rescorla, 1985, 1991), avoidance learning (Declercq & DeHouwer, 2008, 2009a, b), and performance on sustained visual attention tasks (Hirsh & Burk, 2013; Qadri, Reid, & Cook, 2016; Schmajuk & Bushnell, 2009). Likewise, many types of commonly-used conditional discrimination learning and memory tasks may substantially engage occasion-setting processes (Delamater, et al., 2017; Holland, 1992). For example, in the commonly-used DMTS procedure, a sample cue (A) may set the occasion for responding for one subsequently-presented target cue (A), but not another (B). Interestingly, the retention interval between sample and target choice cues interacts with ITI to determine accurate DMTS performance (Roberts & Kraemer, 1987) in the same way that the feature-target interval and ITI interact in occasion setting tasks (“Temporal factors” section). It would be interesting to determine if variables that affect transfer in occasion setting (“Content of occasion setting” section) also affect the generality of DMTS performance, which is often highly specific to the original training cues. Furthermore, it is notable that even a simple discriminated operant training procedure, in which an operant response is reinforced only in the presence of a particular cue, may at times engage occasion setting processes. Indeed, the expression “occasion setting” was coined (e.g., Skinner, 1938) to describe this conditional relation between cue, behavior, and outcome. Interestingly, Davidson, Aparicio, & Rescorla (1988) found that whereas a Pavlovian occasion setter failed to facilitate responding to a separately-trained Pavlovian CS, it facilitated responding to an operant discriminative stimulus, suggesting that instrumental discrimination learning may be more akin to occasion setting than to simple Pavlovian conditioning (Colwill & Rescorla, 1990). Finally, some authors have suggested that even the simplest Pavlovian conditioning procedures may involve occasion setting to some extent. For example, Burns and Domjan (2000) suggested that whereas sign-tracking (cue-directed responding) in quail reflects simple conditioning, goal tracking (food source-directed responding) reflects occasion setting, and Moore and Choi (1998) suggested that the normal timing of responses in eye blink conditioning resulted from an occasion setting process.

It is also important to recognize that there may be multiple modulatory processes, and they may differ from paradigm to paradigm. For example, despite the many parallel findings from appetitive Pavlovian conditioning procedures with rats and Rescorla’s pigeon autoshaping procedures, there appear to be fundamental differences as well. Although temporal variables play a critical role in the acquisition of modulatory powers in rat procedures, they seem largely unimportant in pigeon autoshaping (Rescorla, 1989). Similarly, although in Holland’s Pavlovian preparations a partially extinguished cue is not an adequate target of either positive or negative occasion setters, Rescorla routinely used such a cue as a target of both (“Other eligible targets of occasion setting” section). Interestingly, in Holland’s discrete trial operant procedure, an extinguished cue (but not a partially reinforced cue) served effectively as a target of occasion setting (e.g. Holland, 1991a). Similarly, although Holland demonstrated clear distinctions between negative occasion setting and simple inhibition (“Assays of occasion setting”), Rescorla’s data (1985, 1987, 1989, 1991c) give little evidence for a separate, nonmodulatory inhibitory process. Indeed, Rescorla (1985, 1991c) suggested that a pair of modulatory processes, facilitation and inhibition (which correspond to positive and negative occasion setting) complement a single associative process, excitation.

The suggestion that inhibitory phenomena complement not excitation but facilitation (positive occasion setting) would force a major reevaluation of basic conditioning models, given the substantial role conditioned inhibition has played in those theories (e.g., Pearce & Hall, 1980; Wagner & Rescorla, 1972). Clearly, the development of more precise models of occasion setting (e.g., Delamater, 2012) which specify the interactions between modulatory, configural, and simple conditioning processes, is crucial to the understanding of associative learning. Such models likely would incorporate notions of multiple stimulus representations, both at the level of general properties, as assumed in AESOP (Brandon & Wagner, 1998), and more specific features, as is implicit in accounts such as Bonardi, et al.’s (2017). Finally, it would not be surprising if the rules for interactions among stimulus representations differed from level to level, nor if those interactions went beyond simple arithmetic summation of element strength (e.g., Pearce, 1987). Surprisingly, there has been little investigation of such interactions (e.g. Davidson, et al., 1988; Delamater, et al,, 2010; Rescorla, 1988; vanWijk, Maes, & Vossen, 1997) beyond demonstrations of the independence of the occasion setting and simple conditioning powers of a cue.

Finally, it is important to keep in mind that the phrase “occasion setting”, like other psychological terms, such as “Pavlovian conditioning” or “extinction” can be used to refer to a set of procedures, a set of outcomes, and to a hypothetical process by which those procedures lead to those outcomes. However, the use of a particular procedure does not guarantee the engagement of a particular process nor does it preclude the engagement of other processes. For example, the use of serial compounds may encourage occasion setting at the expense of simple conditioning in many preparations, but the mix of these processes may differ substantially among them. Likewise, variations in response form, transfer, and extinction/counterconditioning effects may occur for reasons other than variations in occasion setting, and may also differ from preparation to preparation. At a minimum, analyses of “occasion setting” should use procedures designed to minimize the contribution of other processes, and multiple assays to distinguish among those processes. This is not an easy task, especially given the ability of different theoretical formulations to account for similar patterns of data. Indeed, perhaps the most important message in this review is that seemingly minor procedural variations (such as stimulus intensity and temporal relations) in associative learning experiments can produce very different outcome patterns, with important translational implications (e.g., amounts of transfer and susceptibility to extinction/counterconditioning). Attempts to apply “principles of associative learning” to therapeutic or other situations must take into account this rich range of outcome patterns. Neurobiological analyses may prove useful in distinguishing among these processes.

Neurobiology of occasion setting

Compared to simple forms of Pavlovian conditioning, there has been remarkably little investigation into the neurobiological basis for occasion setting. Initial studies found that aspiration lesions of the hippocampal formation interfered with the acquisition of serial FP food-based learning (Jarrard & Davidson, 1991; Ross, et al., 1984) but not of an A+, B- discrimination of comparable or greater difficulty. However, ibotenic acid lesions of the hippocampus, sparing fibers of passage, had only transient or no effects with nearly identical training procedures (Holland, et al., 1999; Jarrard & Davidson, 1990, 1991), and lesions of dorsal hippocampus did not affect learning of a serial FP discrimination in fear conditioning (Yoon, Graham, & King, 2011). Nevertheless, Holland, et al. (1999) found that ibotenate lesions of dorsal and ventral hippocampus combined produced substantial impairment in the acquisition of a serial FN discrimination shown by transfer tests to involve negative occasion setting in control rats, and Kanoski, Zhang, Zheng, and Davidson (2010) found that high-fat-diet-induced breaching of the hippocampal blood-brain barrier interfered with acquisition of a similar serial FN discrimination. In addition, hippocampal lesions have been found to impair instances of contextual control thought to involve negative occasion setting, although that evidence is mixed (e.g., Campese & Delamater, 2013; Clarke, Skinner, & van der Kooy, 2001; Holland & Bouton, 1999; Yoon, et al., 2011).

While evidence for a role for the hippocampus in occasion setting is limited, several recent studies have begun to highlight other brain regions important in serial discriminations thought to engage occasion setting. For example, in an extension to the study partially described earlier (“Modulation of incentive motivation”), Fraser and Janak (2018) found that performance in a serial positive patterning discrimination, shown by extinction tests to engage occasion setting, was impaired in rats with temporary inactivation of either the basolateral amygdala (BLA) or orbitofrontal cortex (OFC). Similarly, Bucci and colleagues found that acquisition of serial FN discriminations thought to involve negative occasion setting was impaired by lesions of the retrosplenial (Robinson, Keene, Iiaccarino, Duan, & Bucci, 2011) or prelimbic, but not infralimbic, cortex (MacLeod & Bucci, 2010). Acquisition of that serial FN discrimination was also impaired by chemogenetic stimulation of the nucleus accumbens (NAc), especially when the activity of the OFC was simultaneously depressed (Meyer & Bucci, 2016). And Shobe, et al. (2017) reported modulation of target CS-evoked unit activity within the OFC by a putative negative occasion setter established in a serial FN discrimination in head-fixed mice. Unfortunately, these studies did not include transfer or counterconditioning tests that might be diagnostic of negative occasion setting specifically, as opposed to simpler inhibitory processes.

In the next sections we consider how three of these regions, the BLA, OFC and NAc may participate in a network important for occasion setting and thus might be particularly valuable targets for future research. Anatomically these structures are well-situated to act in concert because the BLA and OFC are reciprocally connected and both the BLA and OFC project to the NAc, which acts as a limbic-motor interface ( Heilbronner, Rodriguez-Ramaguera, Quirk, Groenewegen, & Haber, 2016; Mogenson, Jones, & Yim, 1980; Price, 2007). Based on this circuit architecture there are a number of ways in which these structures might work collectively to retrieve, encode, and update information relevant to an animal’s state and produce adaptive, flexible behavior (Figure 4). Here, we review evidence that suggests a role for each of these structures in these processes, permitting the formulation of testable hypotheses for their collective involvement in occasion setting. In particular, we note how the use of recent technological advances could reveal the brain implementation of occasion setting functions and better inform our understanding of overarching behavioral processes.

Figure 4. Neurobiological substrates of occasion setting.

Sagittal section of a rat brain illustrating regions implicated in occasion setting and their relevant circuitry is indicated by the directional arrows. Green shading indicates regions for which evidence implicates activity within this region as being critical for the acquisition or expression of occasion setting. Other regions shaded in blue are implicated in simple Pavlovian conditioning but a role for these regions in occasion setting remains to be demonstrated. Evidence for the involvement of the dHIPP, shaded in red, in occasion setting is mixed and requires further study. BLA basolateral amygdala, dHIPP dorsal hippocampus, mPFC medial prefrontal cortex, NAc nucleus accumbens, OFC orbitofrontal cortex, PVT paraventricular nucleus of the thalamus, vHIPP ventral hippocampus, VTA ventral tegmental area.

Basolateral Amygdala (BLA)

Lesion and reversible inactivation studies revealed that although normal BLA function is not needed for learning a Pavlovian cue-reward relationship, it is critical for responding appropriately to a cue when the value or significance of that cue or the outcome it signals has changed (Baxter & Murray, 2002; Gallagher & Holland, 1994; Janak & Tye, 2015; Wassum & Izquierdo, 2015). This research demonstrated a critical role for the BLA in a number of associative learning tasks, such as second-order conditioning, reinforcer devaluation, and Pavlovian-to-instrumental transfer (Belova, Paton, & Salzman, 2008; Blundell, Hall, & Killcross, 2001; Burns, Everitt, & Robbins, 1999; Corbit & Balleine, 2005; Everitt, Cador, & Robbins, 1989; Hatfield, Han, Conley, Gallagher, & Holland, 1996; Holland & Gallagher, 2003; Johnson, Holland, & Gallagher, 2009; Málková, Gaffan, & Murray, 1997; Malvaez, et al., 2015; Morrison & Salzman, 2009; Morrison, Saez, Lau, & Salzman, 2011; Parkes & Balleine, 2013; Pickens, Saddoris, Setlow, Gallagher, Holland, & Schoenbaum, 2003; Setlow, Gallagher, & Holland, 2002). Taken together, these findings led to the notion that the BLA integrates internal and external information to generate a state value that is used to facilitate adaptive behavior (Costa, Dal Monte, Lucas, Murray, & Averbeck, 2016; Morrison & Salzman, 2010; Sharpe & Schoenbaum, 2016; Wassum & Izquierdo, 2015). However, the above tasks, which are commonly used to assess the utilization of state value, are not especially amenable to neurobiological investigation, because the phenomena are short-lived or only allow for a few critical test sessions. In particular, the key process the BLA is thought to be critical for, integrating and updating state-related information, typically occurs offline in these tests, especially for those that manipulate outcome value by way of reinforcer devaluation. This precludes an understanding of the ongoing integration and updating of expectations based on cues, internal states, and outcomes in real time, and their immediate use to guide behavior. Additionally, these tasks are not well-suited to neurobiological techniques such as in vivo electrophysiology, which require substantial numbers of samples of behavior and neural activity. In contrast, occasion setting procedures generally generate robust behavioral responses, the value of a cue is updated repeatedly among trials within a session, and the moment-to-moment change in the value of a cue can be controlled within individual trials. Thus, these procedures allow for ongoing and reliable assessments of the neural substrates of such a rapid and integrative process that would be well-suited to test state-value encoding theories of the BLA.

In addition to its role in flexible valuation, the BLA may be critical for some aspects of contextual control. As noted in our discussion of “Context”, the ability of contextual cues to gate responding to explicit CSs may involve occasion setting processes in some circumstances. For example, after pairing of a CS with alcohol, sucrose, shock, or various psychostimulants in one context and extinction of that CS in another context, testing responding to the CS in the training context or another, novel context may renew that responding, compared to testing in the extinction context (Bouton, 2002; Bouton & Moody, 2004; Bouton, Westbrook, Corcoran, & Maren, 2006; Crombag, et al., 2008; Khoo, Gibson, Prasad, & McNally, 2017; Valyear, Villarvel, & Chaudhri, 2017). However, inactivation of BLA at the time of test interferes with this renewal, but does not affect responding if tested in the extinction context (Chaudhri, Sahuque, & Janak, 2008; Chaudhri, Woods, Suhuque, Gill, & Janak, 2013; Fuchs & See, 2002; Fuchs, Evans, Ledford, Parker, Case, Mehta, & See, 2005; Hobin, Goosens, & Maren, 2003; Lasseter, Wells, Xie, & Fuchs, 2011; Millan, Reese, Grossman, Chaudhri, & Janak, 2015; Sciascia, Reese, Janak, & Chaudhri, 2015). Nevertheless, it is difficult to ascertain if these effects reflect a role for the BLA in occasion setting, because the conditions that promote a context’s serving as a positive or a negative occasion setter are unclear. Modeling what is argued to be the key contribution of contexts to conditional responding, their occasion setting properties, may provide clarity into both the ability of these complex cues to promote or prevent renewal and the precise contribution of the BLA in these phenomena.

Orbitofrontal Cortex (OFC)

As with BLA, the OFC is necessary for many complex psychological phenomena that require flexible retrieval of a CS’s value. Disruption of OFC function impairs performance in reinforcer devaluation, PIT, and second-order conditioning tasks ( Gallagher, McMahan, & Schoenbaum,1999; Izquierdo & Murray, 2007; McDannald, et al., 2005; Murray, Moylan, Saleem, Basile, & Tuchi, 2015; Ostlund & Balleine, 2007; Pickens, et al., 2003; Pickens, Saddoris, Gallagher, & Holland, 2005; Ramirez & Savage, 2007; Rudebeck & Murray, 2011; Schoenbaum, Chiba, & Gallagher, 1998; Stalnaker, Cooch, & Schoenbaum, 2015). Some investigators have suggested that whereas the BLA integrates information rapidly and moment-to-moment, the OFC integrates this information from the BLA into a state space that is then used to govern structures downstream from the OFC on a longer time scale (Balleine, Leung, & Ostlund, 2011; Keiflin, Reese, Woods, & Janak, 2013; Rudebeck & Murray, 2014; Sharpe & Schoenbaum, 2016; Wilson, et al., 2014). For example, Wilson, et al. (2014) suggested that OFC plays a critical role in monitoring and updating “hidden state” information that underlies reversal learning. In this view, early instances of reinforcement or nonreinforcement after a reversal might promote hidden state transitions that serve as occasion setters signaling the new reinforcement contingencies, permitting a rapid adjustment of behavior. Thus, the BLA informs the OFC how important a given cue is and the OFC integrates this importance given what has occurred in the recent past, keeping track of what to expect and how to respond in the future. The OFC can then refine how the BLA encodes future information as a result of the reciprocal connections within this circuit. Together, these structures can then generate action through their shared projections to the NAc (Heilbronner, et al., 2016).

These ideas arose from the finding that cue-outcome and response-outcome encoding in the OFC depends on the BLA and that similar encoding in the BLA depends on the OFC (Rudebeck, Mitz, Chacko, & Murray, 2013; Rudebeck, Ripple, Mitz, Averbeck, & Murray, 2017; Saddoris, Gallagher, & Schoenbaum, 2005; Schoenbaum, Setlow, Saddoris, & Gallagher, 2003). We hypothesize that, given this essential role in tracking changes in state value driven by cues over time, the OFC may also be critical for occasion setting. Indeed, Fraser, et al. (Society for Neuroscience Abstract, 2018) found evidence that in the absence of OFC function, rats fail to use occasion setters to modulate conditioned responding. In particular, if the OFC is representing a state space, then we would expect to see transitions and changes in firing across presentations of the feature and target in occasion setting procedures, and the OFC should show significant modulation in response to the omission of the target on feature-alone trials in a F→T+, F−, T− positive patterning discrimination. Identifying these dynamics will require more sophisticated analyses making use of modern computational techniques that are well-suited for comparing the ability to decode such state information from either single neurons or a population. Given that such transitions are experimenter-controlled and can occur numerous times in the course of a single-recording session, the study of occasion setting can lead to a more robust understanding of neural encoding and dynamics within the OFC, as well as how the BLA and OFC interact to support cue-driven modulation of conditioned responding and to state-space theories of the OFC.

Nucleus Accumbens (NAc)

The NAc is classically considered to be an interface between limbic and motor systems, integrating motivational signals and transforming these into behavioral output, and is also the main target of the mesolimbic dopamine system arising from the ventral tegmental area. Much research has revealed a role for the NAc (and NAc dopamine in particular) in a wide range of Pavlovian conditioning phenomena, such as conditioned approach, conditioned reinforcement, and PIT [see reviews by Cardinal, Parkinson, Hall, & Everitt, (2002); Castro, Cole, & Berridge, (2015); Floresco (2015); Nicola (2010); Richard, Castro, Difeliceantonio, Robinson, & Berridge, (2013); and Salamone & Correa (2012), and for details of these results, including important functional differences among subregions and cell populations in the NAc.]

Whereas most research into functions of the NAc have focused on its dopaminergic input, it is important to also recognize the diverse sources of glutamatergic input that likely are responsible for producing activity in NAc neurons. The NAc receives such glutamatergic input from the BLA, OFC and prefrontal cortex, ventral hippocampus, and paraventricular thalamus. Manipulations in each of these regions have implicated them in the control of behavior governed by simple cues (e.g., Bussey, Everitt, & Robbins, 1997; Fitzpatrick, Creeden, Perrine, & Morrow, 2016; Haight, Fraser, Akil, & Flagel, 2015). Although inactivations or lesions of the NAc often are without drastic effect on simple Pavlovian conditioning (Blaiss & Janak, 2009; Chang & Holland, 2013; Chang, Wheeler, & Holland, 2012; Di Ciano, Cardinal, Cowel, Little, & Everitt, 2001; Fraser & Janak, 2017), NAc may play a role in facilitating more complex behaviors. However, there is little evidence for how the NAc integrates both dopaminergic and glutamatergic inputs to spur behavioral responses when, for example, cues are ambiguous and their significance must be rapidly resolved. Exploiting occasion setting preparations while simultaneously monitoring axon terminals from a selected glutamatergic input and activity in dopamine terminals, or dopamine release itself, might resolve how the NAc integrates internal and external signals over time to produce appropriate conditioned responding. Given the NAc is conceptualized as a primary region responsible for Pavlovian reward-seeking, occasion setting tasks are an easy-to-implement addition to batteries of behavioral tasks that could shed light on more complex contributions of the NAc to cue-triggered behavior. The advent of technologies to monitor projection-specific activity with fiber photometry coupled with calcium- or dopamine-sensitive sensors overcomes prior technical obstacles associated with electrochemical and biosensor detection methods, as both of these signals can be extracted simultaneously from an easy-to-construct and robust optic fiber implant (Kim, et al., 2016; Lerner, et al., 2015; Patriarchi, et al., 2018). We anticipate such approaches to shed new light into both dopamine modulation of learning and motivation as well as new theories of NAc function in reward-seeking.

Implementing novel neuroscience technologies to understand dynamical cue-driven processes

Combining occasion setting procedures with in vivo electrophysiological recordings or cell-type and circuit-specific calcium imaging would allow for an understanding of neural processes that would support such dynamic cue-driven behavior. We are at a time in neuroscience research in which we need to subject more complex behavior to analysis with tools that provide understanding of diverse functions and encoding of information within neural circuits (Krakauer, Ghazanfar, Gomez-Marin, MacIver, & Poeppel, 2017). While there are many possible behaviors to examine, the behaviors used most commonly with these techniques might not best make use of these tools to address the diversity of functions a circuit may achieve. For instance, it is difficult to extrapolate from the ability of a given circuit or region to support optogenetic self-stimulation to that circuit’s participation in other psychological processes (e.g., Saunders, Richard. Margolis, & Janak, 2018). Occasion setting is not much more difficult to implement as a behavioral preparation than simple operant or Pavlovian conditioning, and there are numerous aspects of this behavior, which we have emphasized in this review, that make it well-suited for understanding the maintenance, retrieval, and updating of the significance of numerous cues and their relationships with each other and reinforcers over extended timescales. Here, we describe some recent advances in neural recording and manipulation that we believe are well-suited to incorporation with occasion setting procedures to understand such dynamical encoding.

Serial discrimination procedures that yield occasion setting have numerous well-defined time periods for optogenetic manipulations that would allow for dissection of the time-windows in which neural activity within a region, cell type, or circuit is critical to properly update and retrieve cue-guided expectations in the production of adaptive behavior. For example, time-locked optogenetic manipulations can be made during only the occasion setting feature, only the target CS, in the gap between their presentation, or during a number of combinations of those intervals. Taking advantage of the ability to restrict neural manipulations in time would resolve when activity in a circuit or region is critically transmitting or retaining forward-looking modulation predicted by the occasion setter. For instance, is dopamine activity critical only at the time of the occasion setter or the target CS? Is it possible to condition optogenetic activation of dopamine neurons to act as an occasion setter or a target CS? Could occasion setting be acquired by a cue that predicts dopamine neuron stimulation? Time-locked and controlled activation and inhibition will provides clues to the underlying computations supporting occasion setting, and coupled with evidence from neural recordings, these findings could lead to new theories and models of reinforcement learning that better account for the nuances of cue-guided behavior in the real world. Occasion setting procedures may likewise help overcome issues that are related to deconstructing the influence of contexts on behavior in a variety of preparations, because occasion setting cues can be phasic, localizable, and discrete events that are suitable for addressing how such a model “context” could act to alter both neurobiology and behavior, within the same session and same animal.

One outstanding question is what regions and circuits are critical for the acquisition and expression of occasion setting. Previous work suggests that the OFC and NAc are involved in the acquisition of occasion setting, whereas the BLA and OFC are essential for its expression. However, it is possible that these previous studies manipulating entire brain regions obscure more fine-grained changes in circuit function over time (e.g., Do-Monte, Quinones-Laracente, & Quirk, 2015; Do-Monte, Quirk, Li, & Penzo, 2016). Perhaps early in acquisition the projections from either BLA or OFC to the NAc are critical, but with further training the BLA and OFC can support the process independent of the NAc. Since the invention of efficient retrograde-travelling AAVs (Tervo, et al., 2016), chemogenetic manipulations of defined pathways can be achieved in many circuits in wild-type animals. These tools are powerful because they allow for either repeated manipulation of activity throughout acquisition and then later in expression in the same animal, or manipulations across defined time points from early to late training. These methods are less invasive than intracranial infusions, less likely to produce damage when repeatedly applied, and can allow for simultaneous dissection of numerous circuits with (for example) the muscarinic-based DREADDs hM3Dq or hM4Di and the kappa opioid-based DREADD KORD, each of which have their own selective ligand (Alexander, et al., 2009; Gomez, et al., 2017; Marchant, et al., 2016; Vardy, et al., 2015). This approach can allow for simultaneous excitation or inhibition of (for example) either BLA projections to the OFC and NAc during acquisition, or both OFC and BLA projections to the NAC after animals are well trained.

While traditional tetrode and electrode bundle recording methods are standard and well-suited for answering numerous questions regarding neural encoding, the availability of new high-density silicon probes with thousands of recording sites per probe allows for the ability to simultaneously record activity from hundreds to thousands of neurons across diverse brain regions (Jun, et al., 2017). Although freely-moving behavior is critical for allowing animals to behave more naturalistically, adapting occasion setting for head-fixed preparations would also allow high-throughput neuroscientific analyses such as 2-photon imaging, mesoscale imaging, and numerous acute optogenetic-assisted electrophysiological recordings, all of which have been done with simple Pavlovian conditioning (Beyeler, et al., 2016; Otis, et al., 2017; Sofroniew, Flickinger, King, & Svoboda, 2016).

Further insight into the neurobiology of occasion setting has the potential to inform current theoretical debates regarding the nature of the representations that support occasion setting. For instance, in a serial FP occasion setting preparation, one might ask how the target representation changes depending on the presence or absence of the feature. Configural theories might expect defined sets of neurons to respond selectively to the target only on compound trials, that is, a discrete neuronal population which codes the feature-target combination. By contrast, more frequent observation of neurons that respond on target-alone trials, but show significant modulation of that responding when the target is presented within the feature+target compound, might suggest modulatory coding as a primary way for achieving occasion setting. Similarly, determining how the presentation of the feature alters neural activity may help resolve the manner in which an occasion setter acts. For instance, feature presentation might alone produce a change in the population dynamics of a given region while the change in firing in response to target presentation stays stable, suggesting the feature alone is able to gate general expectations of reinforcement. Or perhaps the feature evokes little or no prolonged change in neural activity but can alter the strength of responding to target presentation, suggesting it gates the target’s representation. In this instance, it would be ideal to make use of high-density silicon probes to monitor hundreds of neurons across numerous brain regions simultaneously to better understand how not just one region, but coordinated activity across, say, the amygdala, thalamus, striatum, and cortex evolve based on expectations of future cues and outcomes. Apart from high-density recordings, making use of machine learning approaches to better classify and identify stimulus-triggered changes at the population level that may not be evident based on firing rate changes could be useful in evaluating the degree to which a given region is modulated by an occasion setter.

Beyond electrophysiological approaches that are typically blind to cell- and circuit specificity, one- and two-photon imaging approaches are becoming standard in systems neuroscience and their implementation within operant and Pavlovian conditioning tasks is increasingly common. These techniques would allow for identification and imaging of a large subset of neurons in a defined circuit or cell-type specific manner. In freely moving rodents, the miniaturization of lasers and microscopes has allowed for more naturalistic settings (Ghosh, et al., 2011; Resendez, et al., 2016). In the past year, these miniaturized one-photon head-mounted endoscopes have advanced to become tetherless, making their use in larger rodents like rats, where head torque is a problem for maintaining high quality data, much more likely (Liberti, Perkins, Leman, & Gardner, 2017). These miniature microscopes are especially useful for monitoring a large subset of neurons consistently across many days, making it possibly to stably record and infer changes in neural dynamics from early to late stages of learning (Grewe, et al., 2017). As mentioned above, combining imaging with a miniature endoscope with circuit specific-labeling could inform whether the same region, projection, or cell-type is critical at early or late timepoints. It has been recently shown that in Pavlovian fear conditioning (Grewe, et al., 2017), the overall population code in response to CS presentation becomes similar to that evoked by the footshock US. There may be intriguing differences in how population dynamics change when outcome ambiguity is introduced, and when that ambiguity is resolved by the presentation of occasion setters.

Finally, because it is easily implemented in laboratories that are already using optogenetics, fiber photometry has become increasingly popular. Fiber photometry is the recording of bulk fluorescence from a defined cell-type, circuit, or brain region depending on the selective expression of an indicator (Calipari, et al., 2016; Gunaydin, et al., 2014; Kim, et al., 2016; Lerner, et al., 2015). This approach permits recording activity from selected axon terminals, because many of these fluorescent indicators are highly expressed throughout neuronal processes. Indicators are becoming increasingly available beyond the calcium-sensitive GCaMP, with the recent invention of sensitive and selective acetylcholine and dopamine fluorescent sensors (Jing, et al., 2018; Patriarchi, et al., 2018). Additionally, by exploiting differences in emission and excitation spectra for various activity indicators, like GCaMP and jRGECO (Chen, et al., 2013; Dana, et al., 2016), one can perform two-color fiber photometry that permits more complete analysis of information flow and integration within brain circuits. For example, multisite two-color fiber-photometry could be used to record both cell-body and axonal fluorescence of BLA and OFC simultaneously to better understand the flow and evolution of information in this reciprocal circuit. The utility of fiber photometry to record activity in both cell bodies of the OFC and BLA, as well as OFC and BLA terminals in each region, allows for unprecedented understanding of how activity in these reciprocal circuits evolves with learning.

Collectively, we feel that a transition from understanding simple Pavlovian conditioning to a neurobiology of occasion setting will better inform understanding of brain-behavior relations. Occasion setting procedures are simple to implement and may lead to a richer understanding of how cue-triggered behavior occurs and better elucidate how brain regions and circuits contribute to cue-reward learning. Tapping into uncertainty and ambiguity allows for a better understanding of whether a given brain region conveys an average value signal related to a cue, a state space or state value related signal, cue- or outcome-prediction errors, as well as the ability to record the evolution of neural dynamics across well-defined timescales. Taken together, we believe that occasion setting tasks can better model how cues interact in the real world to promote and inhibit reward-seeking. Given that the ability to flexibly respond in the face of an uncertain and changing environment is dysregulated in a number of psychiatric illnesses, occasion setting might prove to be a more useful model for elucidating neurobiological targets for treatments than simple conditioning models.

Conclusion

We have provided a comprehensive review of occasion setting, from variables influencing its acquisition to its little-known neurobiological bases. Clearly, the field of associative learning has much to explore. Simple variations in the temporal and other properties of stimuli can have profound effects on what is learned. Ignoring these differences in the content of learning has yielded incomplete models of cue-driven behavior and its neurobiological underpinnings. There is much room for analysis of occasion setting at many levels, from computational approaches to accommodate occasion setting within standard reinforcement learning models, to a more thorough understanding of the neurobiological substrates of occasion setting and their relation to those underlying other cue-driven phenomena. Given the many recent advances in neuroscientific methodology, we especially hope to spur new research in this latter domain.

Updating our models of cue-driven behavior to include occasion setting and other processes to represent conditional information about future events can better mirror the ambiguity and modulation of cue-triggered behavior in the real world. For example, in chronic relapsing disorders like addiction, there are numerous complex processes that likely regulate the motivational salience of drug-associated cues that ultimately trigger relapse. It is well known that relapse occurs even after long periods of abstinence and even after treatment in the clinic, but much research and theory has still focused on simple properties of cues and their regulation by contexts. However, it has been difficult to capture how to appropriately translate context and alternative-reinforcer based approaches to produce novel and effective therapies beyond what is currently available. We view the occasion setting approach as one that can appropriately capture the moment-to-moment regulation and transformation of neutral cues into powerful triggers for drug-seeking that are lacking in current models. In addition, it is likely that the two main physiological adaptations that occur following drug use, tolerance and sensitization, are both under the control of occasion setting processes (Anagnostaras & Robinson, 1996; Agnagnostaras, et al., 2002; Ramos, et al., 2002). Thus, adopting occasion setting procedures can provide insight into psychological mechanisms supporting drug-seeking, physiological responses to drugs of abuse, and interactions between these two domains. The time is ripe for occasion setting to become a standard approach to understanding learning and motivation.