Determinants of cue interactions

Abstract

In a Pavlovian conditioning situation, there are many training variables that may affect responding to a conditioned stimulus (CS), such as contiguity, contingency, and the presence of other CSs. This review describes recent experiments that show that some manipulations that usually decrease responding to a CS may have the opposite effect when they are combined with other normally pernicious manipulations. A theoretical framework that explains these so-called counteraction effects is provided. The apparent boundary conditions on the effects and limitations of the theory are discussed.

There are a large number of variables that affect basic Pavlovian conditioning, such as contiguity, contingency, conditioned stimulus (CS) duration, amount of training, the spacing of trials, and the number of stimuli present on a given training trial. Researchers have extensively studied the effects of each of these individual variables on learning, but they have paid less attention to the potential interactions between these variables. The effects of select parameters which attenuate stimulus control of acquired behavior, such as degraded contingency (e.g., Rescorla, 1968), long CS duration (e.g., Gibbon et al., 1977), massed training trials (e.g., Barela, 1999), and overtraining (e.g., Kamin, 1961), have all been explained by at least some theories of learning through an appeal to associative competition between the target stimulus and the experimental context. In contrast, attenuated stimulus control due to the presence of nontarget stimuli on target stimulus training trials has been attributed to associative competition between stimuli. Recent work in our laboratory has focused on studying the ways that these two types of competition with a target stimulus might interact.

Nonreinforced exposure of a potential CS retards the emergence of conditioned responding when the stimulus is subsequently paired with an unconditioned stimulus (US). This phenomenon has been referred to as the CS-preexposure effect or latent inhibition (e.g., Lubow, 1973; Lubow and Moore, 1959). Latent inhibition is commonly observed in many of the different preparations used to study associative learning (Lubow, 1989; Lubow and Gewirtz, 1995). Different mechanisms have been proposed to account for the effect, but all make the common prediction that extensive CS preexposure should retard the emergence of conditioned responding when the CS is subsequently paired with the US. For example, this retardation of conditioned responding could be regarded as consequence of a diminution in the CS–US contingency (Escobar and Miller, 2004) or a decrease in attention to the CS (Lubow, 1989). In a typical latent inhibition design, the CS that is preexposed is the same CS that is reinforced and tested. However, a number of studies have examined the effect of target CS preexposure on subsequent compound conditioning in which the target CS is accompanied by a companion stimulus (e.g., Nakajima et al., 1999; Schnur, 1971). Some of these studies have produced unexpected results, specifically increased behavioral control by the preexposed stimulus (e.g., Blaisdell et al., 1998; Ishii, 1999; Loy and Hall, 2002).

Typically, reinforcing a compound stimulus that is comprised of CSs of different saliences results in overshadowing of the less salient CS. Overshadowing is evidenced by less conditioned responding to the less salient CS relative to a situation in which the same CS is trained without a companion (e.g., Pavlov, 1927). One might expect, as many theories of learning do, that latent inhibition and overshadowing would summate and produce a response deficit that is more profound than either of the effects alone. However, Blaisdell et al. (1998) observed the opposite effect in a conditioned suppression preparation. Subjects that received both treatments showed responding that was similar to that exhibited by subjects that received neither of the two individually response-degrading treatments. Thus, nonreinforced CS exposure before conditioning counteracted the overshadowing effect. Alternatively stated, the presence of a more salient companion CS during conditioning counteracted the latent inhibition effect. This sort of counterintuitive interaction between normally response-degrading treatments has been called a counteraction effect. A similar effect has been observed by other researchers. For example, Ishii (1999) observed a counteraction between latent inhibition and overshadowing in a conditioned suppression preparation, but the counteraction was asymmetrical. In his Experiments 2 and 3, subjects that received target CS preexposure prior to compound conditioning trials showed an attenuation of latent inhibition relative to subjects that received CS preexposure before reinforcement of the target CS alone. However, CS preexposure before compound conditioning did not produce responding greater than that observed in subjects that received compound conditioning with no CS preexposure. Although the counteraction was asymmetrical, there clearly was no summation between latent inhibition and overshadowing.

1. The extended comparator hypothesis

The counteraction between latent inhibition and overshadowing is explicable in terms of Denniston et al.'s (2001) extended comparator hypothesis (ECH). The ECH is a model that focuses on the expression of acquired associations. It is not a model of acquisition, but it does assume that noncompetitive acquisition of associations develops between contiguous stimuli, and an established CS–US association may be subsequently extinguished when the CS is present without the US. That is, the formation of a CS–US association will not be impeded by the presence of other stimuli, but it can be degraded through extinction. The ECH posits multiple comparator processes that interact with each other and ultimately determine responding to a CS at test. Its basic intuition is that responding to a CS is not determined solely by the degree that a CS predicts a US, but instead the degree to which the CS predicts a change in the likelihood of the US relative to other CSs (e.g., context) that were present during training. In conjunction with a noncompetitive stochastic learning rule (e.g., Bush and Mosteller, 1955), the ECH can account for a wide variety of learning phenomena by focusing on mechanisms that are active at the time of testing. In order to understand the ECH, it is useful to consider a single iteration of the individual comparator processes that compose the model.

Fig. 1 shows a simple comparator process that might occur when a target CS is tested after it has been reinforced in compound with another CS, as in a stimulus overshadowing preparation. The target stimulus directly activates a representation of the US, which encourages conditioned responding. The strength of the directly activated US representation is dependent on the strength of the association between the target and the US (Link 1 in the framework of the model). This mechanism for conditioned responding is rather common among associative models (e.g., Rescorla and Wagner, 1972; Wagner, 1981). Additionally, the target stimulus also directly activates a representation of its companion stimulus (called a comparator stimulus) that was present during reinforcement. The strength of the representation of the comparator stimulus is dependent on the strength of the within-compound association between the target and the comparator stimulus (Link 2). Then the representation of the comparator stimulus activates its own representation of the US based on the strength of the comparator representation and the strength of the comparator–US association (Link 3). This representation of the US is referred to as the indirectly activated US representation because it is not directly activated by the target CS. The strength of the indirectly activated US representation reduces excitatory responding to the target stimulus and encourages behavior indicative of conditioned inhibition. Specifically, excitatory responding to the target is facilitated by the strength of Link 1, but reduced by the product of Links 2 and 3.¹ Note that there are no inhibitory associations. Behavior indicative of inhibition arises as a result of the relative values of different excitatory associations (see Denniston and Miller, 2007).

Fig. 1.

The comparator hypothesis. Boxes represent observable events (i.e., stimulus and response). Ovals represent mental representations of stimuli. The diamond represents the comparator process.

The ECH can explain both overshadowing and latent inhibition. Applied to overshadowing, comparator theory assumes that the target CS at test activates a representation of the US, which positively contributes to conditioned responding (Link 1). The target also activates a representation of the overshadowing CS (Link 2), which in turn activates its own representation of the US (Link 3). If Links 2 and 3 are sufficiently strong, responding will be noticeably attenuated relative to the absence of the overshadowing CS during training. If the overshadowing CS is more salient than the target, then Link 3 should be stronger than Link 1, and overshadowing should be robust. Comparator theory has a slightly different explanation of latent inhibition because there is no punctate stimulus present to serve as a comparator in most demonstrations of latent inhibition. Instead, the context must serve as the competing CS in this situation. Although some theorists consider latent inhibition to be a result of an attentional mechanism (e.g., Lubow et al., 1976; Mackintosh, 1975), there is evidence that supports the notion that it is caused by competition with the context (e.g., Grahame et al., 1994). According to comparator theory, CS preexposure augments the strength of Link 2 (i.e., the context–CS association), but not of Link 1. When the CS is later tested after reinforcement, responding is reduced relative to a situation in which Link 2 was not enhanced by preexposure. If the context differs between preexposure to training, or if the context is extinguished after preexposure, Link 2 will not be enhanced at test and latent inhibition should be negligible. Consistent with this account, latent inhibition is context specific in that it is reduced if the context of preexposure differs from the context of conditioning (e.g., Lovibond et al., 1984). Furthermore, latent inhibition is attenuated if the context is extinguished between preexposure and testing (e.g., Escobar et al., 2002a; but see Hall and Minor, 1984) or between conditioning and testing (Grahame et al., 1994), which indicates that a strong association between the CS and the context of reinforcement mediates the effect. Although it is certainly possible that latent inhibition could also be caused by changes in attentional processing in certain situations, it is at least in part a result of cue-competition between the CS and the context (for a review, see Escobar et al., 2002b).

In order to explain the counteraction between latent inhibition and overshadowing, the ECH assumes that a target stimulus can have multiple comparator stimuli whose representations are simultaneously activated at test. Moreover, these comparator stimuli compete with each other. If a target stimulus (X) is presented (preexposed) without reinforcement before it is reinforced in compound with a more salient stimulus (A), there are two comparator stimuli that have the potential to reduce responding to X at test: the context and A. These stimuli are referred to as first-order comparator stimuli in the framework of the model. If separately paired with X, multiple first-order comparator stimuli have a net summative effect in suppressing responding to the target (Grahame et al., 1992). However, in the situation of overshadowing and latent inhibition, the overshadowing stimulus and the context were present simultaneously because compound training occurred in the same context in which the target was pre-exposed. The model includes an additional tier of comparator processes that allows the effectiveness of first-order comparator stimuli to be modified by second-order comparator stimuli provided there is an association between the first- and second-order comparators. Thus, the overshadowing stimulus not only acts as a first-order comparator stimulus for the target, but, through its action on the context, it also acts as a second-order comparator stimulus. Similarly, the context acts as a first-order comparator stimulus and also, through its action on the overshadowing stimulus, it acts as a second-order comparator stimulus.

Because CS-preexposure and overshadowing treatments produce two first-order comparator stimuli for X, there are two groups of comparator processes that occur when X is tested. For the sake of simplicity, we will first consider the comparator processes in which A acts as the first-order comparator stimulus (see Fig. 2). The presentation of X directly activates a representation of the US based on the strength of Link 1 (i.e., the association between X and the US). Depending on the strength of Link 2.1 (i.e., the within-compound association between X and A), X also directly activates a representation of A, which through Link 3.1 (i.e., the A–US association) activates its own representation of the US. The product of the strengths of Links 2.1 and 3.1 reduce the responding to X that is otherwise determined by the strength of Link 1. The additional tier of comparator processes allows second-order comparator stimuli to modulate the effectiveness of first-order comparator stimuli, thus affecting the strengths of Links 2.1 and 3.1. The left panel of Fig. 2 shows the comparator process that affects Link 2.1. In this comparator process, X directly activates a representation of A based on the strength of Link 2.1 (Link 2.1 is the same as Link 2 in the original comparator hypothesis). But X also activates a representation of the context based on the strength of Link 2.2, which should be strong if X is preexposed in the training context before conditioning. As conditioning of A occurred in the same context as preexposure, the context activates its own representation of A based on the strength of its association with A (Link 2.3). The products of Links 2.2 and 2.3 reduce the activation of A through Link 2.1, impairing A's ability to overshadow X. By reducing the activation of the A representation, preexposure of X in the training context attenuates overshadowing by A. Because the context also acts as a first-order comparator for X, there is another group of comparator processes that occur when X is tested. In this second group of comparator processes, the roles of A and the context are reversed relative to Fig. 2. As a result, the function of A as a second-order comparator stimulus attenuates the effectiveness of the context to produce latent inhibition as a first-order comparator stimulus. Thus, each first-order comparator stimulus for X simultaneously serves as a second-order comparator stimulus. Although the indirectly activated US representation will be the sum of these two comparator processes, the overall effect of the first-order comparator stimuli is reduced by the second-order comparator effects that they exert on each other. In this way, the ECH predicts that overshadowing and latent inhibition will counteract each other.

Fig. 2.

The extended comparator hypothesis applied to CS preexposure followed response). Ovals represent mental representations of stimuli. Diamonds represent comparator processes.

2. Other counteraction effects

The ECH predicts many counteraction effects, and most of those tested to date have involved overshadowing and a context-mediated effect that is evidenced by a loss of conditioned responding. There are multiple examples of counteraction effects in the literature, many of which are detailed below. Except where otherwise noted, the following experiments used a conditioned suppression preparation with rats as subjects in which auditory and visual stimuli served as CSs and a footshock served as the US. Reinforcement of the target CS in the presence of another CS is referred to as compound training, and reinforcement of the target CS alone is referred to as elemental training. The overshadowing CS was always more salient than the target CS to facilitate overshadowing.

2.1. Partial reinforcement and overshadowing

Of the counteraction effects that have been observed, the counteraction between partial reinforcement and overshadowing observed by Urushihara and Miller (2007) is perhaps most similar to the counteraction between latent inhibition and overshadowing. In a factorial design, they manipulated the reinforcement schedule (continuous or partial) and the type of training (elemental or compound). They observed a reduction in conditioned responding when subjects were given either compound training with continuous reinforcement (i.e., overshadowing) or elemental training with partial reinforcement (achieved by adding nonreinforced presentations of the CS). In contrast, subjects that received elemental training with continuous reinforcement (i.e., simple acquisition) or compound training with partial reinforcement (i.e., AX+ and AX− trials) exhibited greater responding. This latter situation demonstrates a counteraction effect. In other words, the normally response-degrading effect of partial reinforcement appeared to counteract overshadowing and vice versa. The ECH accounts for this counteraction effect similar to the way that it accounts for the counteraction between overshadowing and latent inhibition. The addition of nonreinforced compound trials strengthened the A–X within compound association as well as the A–context and X–context associations relative to subjects that received only the reinforced compound presentations. This allowed the context to act as an effective second-order comparator stimulus for the target CS, thereby reducing overshadowing by weakening the potential of the target to activate a representation of the overshadowing CS. Likewise, the overshadowing CS presumably reduced the effectiveness of the context to act as a first-order comparator stimulus. The mechanism underlying this counteraction between partial reinforcement and overshadowing appears to be very similar to the counteraction effect observed by Blaisdell et al. (1998) because they both involve strengthening of a target CS–context association that interacts with the association between the target CS and the overshadowing CS in such a way that augments responding relative to the appropriate control groups in which the target has only one effective comparator stimulus.

2.2. Degraded contingency and overshadowing

Blaisdell et al. (1998) and Urushihara and Miller (2007) both investigated counteractions between overshadowing and treatments that involve nonreinforced CS exposure in the training context. Nonreinforced exposure of a CS is one of the two ways to operationally degrade the contingency between a CS and a US. Other studies have investigated the potential for US-alone exposure to counteract overshadowing. Urcelay and Miller (2006) conducted a series of experiments that investigated the potential interaction between compound training and unsignaled US exposure. Unsignaled exposure to the US during CS–US training can result in a reduction in conditioned responding that is typically referred to as the degraded contingency effect (e.g., Rescorla, 1968). Urcelay and Miller demonstrated that the degraded contingency effect is attenuated by compound training, and the overshadowing effect that typically occurs with compound training is reduced by unsignaled US presentations. In other words, they observed a counteraction between degraded contingency and overshadowing. The ECHs account of this phenomenon is slightly different from that of counteraction between latent inhibition and overshadowing. According to the model, unsignaled US presentations cause the context to gain excitatory strength, which allows the context to serve as an effective comparator stimulus for an elementally trained target CS because Link 3 is enhanced (e.g., Grahame et al., 1992). If unsignaled USs are interspersed among compound training trials, the context's potential to reduce responding to the target CS will be attenuated by the overshadowing CS, which should act as a strong second-order comparator stimulus for the target (as well as a first-order comparator stimulus). Likewise, the overshadowing CS will have a limited effect as a first-order comparator for the target because the context's strong association with the US will cause the context to serve as an effective second-order comparator that attenuates the value of the overshadowing CS as a first-order comparator. Therefore, the two treatments should counteract each other, resulting in relatively unimpaired strong responding to the target CS. Unlike the counteraction effects that involve CS-alone exposure, this counteraction relies on an enhancement of the association between the context and the US rather than the association between the target and the context.

2.3. US preexposure and overshadowing

Urushihara and Miller (2006) also studied the effect of unsignaled US exposures on overshadowing. Instead of interspersing USs with CS–US pairings, they preexposed the USs in the training context before reinforced compound or elemental training. US preexposure appeared to attenuate responding to the elementally trained CS, an effect that is widely regarded as blocking by the context (e.g., Ayres et al., 1985). In contrast, US preexposure enhanced responding to a CS that otherwise would have been overshadowed by its companion. The ECH accounts for this effect in the same way that it accounts for the counteraction between degraded contingency and overshadowing. Because the context builds a strong association with the US, it becomes a more effective first-order comparator stimulus for an elementally trained CS and a more effective second-order comparator for an overshadowed CS.

2.4. Trial massing and overshadowing

In the aforementioned results, overshadowing interacted with a variety of treatments that operationally degrade the CS–US contingency and strengthen the association between the context and the CS (Link 2) or the US (Link 3). There are other ways to manipulate these associations without adding or subtracting CSs and or USs. For example, the behavioral control evoked by a CS that is trained elementally tends to be directly related to the spacing of the conditioning trials. When trials are temporally massed, conditioned responding is generally weaker than in a situation in which trials are spaced (e.g., Barela, 1999). The reduction in conditioned responding may be referred to as a trial-massing effect. However, the relationship between trial spacing and conditioned responding to a CS appears to be reversed if the CS is reinforced in compound with another CS. Stout et al. (2003a) found weaker responding to a CS that was reinforced with an average intertrial interval (ITI) of 40 s relative to a CS that underwent similar training with an average ITI of 960 s. However, responding was stronger with shorter rather than longer ITIs if the CS was trained in compound with another CS.

In order to explain the effect of trial massing and the counteraction between the trial-massing effect and overshadowing, the ECH assumes that responding to a CS is weakened by trial massing because the context has a relatively strong association with the CS and US compared to otherwise equivalent training with spaced trials. With more temporally spaced trials, the associations between the context and the CS and US should be weakened by extinction that occurs during the intertrial interval (for evidence of the context's role in the trial-massing effect, see e.g., Yin et al., 1994). Thus, for an elementally trained CS, the strengths of Links 2 (CS–context) and 3 (context–US) are greater when trials are temporally massed. In contrast, the trial-massing effect is reversed for an overshadowed CS. According to the model, this occurs because massed training allows the context to form a stronger association with the overshadowing CS and work as an effective second-order comparator for the target CS. As a second-order comparator, the context attenuates the potential of the overshadowing CS to act as a first-order comparator. In turn, the overshadowing CS reduces the effectiveness of the context to act as a first-order comparator for the target. It is important to note that this counteraction effect relies on variations in both Links 2 and 3, whereas the previously discussed counteraction effects focused on only one of these two links.

2.5. CS duration and overshadowing

Like trial spacing, CS duration is a timing variable that is known to affect responding to an elementally trained stimulus. Increasing the CS duration causes a reduction in conditioned responding that is very similar to the reduction caused by decreasing the trial spacing (e.g., Gibbon et al., 1977; Gibbon and Balsam, 1981). The observation that longer CSs tend to provoke less conditioned responding relative to shorter CSs when the CS is trained alone could be referred to as the CS-duration effect. Urushihara et al. (2004) investigated whether this relationship between CS duration and conditioned responding is any different if a CS is trained in compound with another CS. In a factorial design, they administered elemental or compound training with either long (25-s) or short (2.5-s) CSs. They found that responding was weak when the subjects received elemental training with a long CS (i.e., the CS-duration effect) or compound training with a short CS (i.e., overshadowing). However, subjects that received compound training with a long CS or elemental training with a short CS showed stronger responding, the former of which is a counteraction effect and the latter is basic acquisition. In a subsequent study, Urushihara and Miller (2007) have replicated this counteraction effect.

The ECH accounts for the counteraction between the CS-duration effect and overshadowing by appealing to the strong CS–context associations that seemingly develop when long CSs are used during training (i.e., longer co-presence of the CS and context should strengthen the association between them). For an elementally trained CS, a longer duration results in an enhancement of the association between the CS and the context (Link 2). This enhancement generally augments the context's potential to act as a first-order comparator. In a compound-training situation, long CSs allow the context to form strong associations with both the target CS and the overshadowing CS. The augmentation of Links 2.2,2.3, and 3.2 causes the context to act as an effective second-order comparator stimulus, which should reduce the effectiveness of the overshadowing CS as a first-order comparator. In addition, the overshadowing CS acts as an effective second-order comparator stimulus, which should reduce the effectiveness of the context as a first-order comparator. This counteraction effect differs from the counteraction between trial massing and overshadowing because the strength of the context–US association is not manipulated.

2.6. Overtraining and overshadowing

In addition to contingency and timing variables, there is evidence that the number of trials can differentially affect compound and elemental training. Extensive CS–US pairings that are administered after stimulus control of behavior has reached an asymptotic level can actually result in a reduction in conditioned responding. This overtraining effect has often been observed in situations that involve elemental training (e.g., Kamin, 1961; Millenson and Hendry, 1967; Miller et al., 1981). In contrast, in a compound conditioning situation Stout et al. (2003a; also see Bellingham and Gillette, 1981) showed that extensive training can attenuate overshadowing, in that it results in increased responding to a target CS trained in compound relative to a target CS given fewer compound conditioning trials. The ECH anticipates this difference as a result of increases in all of the associations that involve the context with extensive training. The model assumes that the context is appreciably extinguished between trials if they are sufficiently spaced, but this extinction progresses more slowly than excitation is accrued.² Over repeated trials, the context, despite its presumably low associability, can eventually build strong associations with both the CSs and the US. In an elemental training situation, overtraining allows the context to act as an effective first-order comparator stimulus because Links 2 and 3 are slowly developed until they are strong. In a compound training situation, overtraining allows the context to build a strong association with the overshadowing CS as well as the target CS. This allows the context to serve as an effective second-order comparator stimulus and attenuate overshadowing. As with the other counteraction effects, the overshadowing CS also serves as an effective second-order comparator and attenuates the context's potential to reduce responding to the target CS.

It should be noted that Stout et al. (2003a) did observe a reduction in overshadowing with extended training, but there was only a tendency (not statistically significant) towards an overtraining effect in their elementally trained control groups. Recent studies from our laboratory (Urcelay et al., under review) have indicated that a robust overtraining effect occurs with 50 training trials, which is somewhat larger than the 36 trials used by Stout et al. Moreover, Urcelay et al. used a 30-s CS whereas Stout et al. used a 5-s CS. Therefore, it is possible that the context acted as an effective second-order comparator in the compound-training situation before it acted as a strong first-order comparator in the elemental-training situation. This is possible in the framework of the ECH because A was more salient than X, which would cause the context to build associations more rapidly with A than X.

2.7. Overtraining and overexpectation

In addition to observing a counteraction between overtraining and overshadowing, Urcelay et al. (under review, Experiment 3) studied the interaction between overtraining and overexpectation. Overexpectation refers to a decrease in responding to two independently reinforced CSs after they have been compounded and reinforced together (e.g., Blaisdell et al., 2001; Rescorla, 1970). Urcelay et al. trained two CSs independently, and then compounded the two CSs and reinforced them together. With a few reinforced compound trials, an overexpectation effect was observed. But with many reinforced compound trials, the overexpectation effect vanished. Importantly, control subjects given many reinforced elemental trials exhibited the overtraining deficit in responding. Thus overtraining and overexpectation treatments appear to counteract each other. The overtraining effect presumably is due to the context, acting as a comparator for the target CS, becoming highly excitatory with extensive reinforcement. The overexpectation effect is presumably due to each CS becoming a comparator for the other CS. Thus, the counteraction is again seen to arise from two comparator stimuli; the context and the other CS, undermining the effectiveness of each other. Additionally, the strong conditioned responding in the subjects that received many compound reinforced trials precludes an account of the overtraining based on habituation to the US occurring as a result of its many presentations.

2.8. Blocking and blocking

All of the counteraction effects that have been discussed to this point have involved training the target stimulus in situations with two potential competing CSs: an overshadowing CS and the training context. This suggests that counteraction effects might be constrained to such situations. For example, it seems especially dubious to anticipate a counteraction effect when a target CS is trained with two overshadowing CSs or two blocking CSs. However, Witnauer et al. (in press) have recently observed the latter. They compared a typical blocking treatment, in which a target stimulus was reinforced in compound with a previously reinforced CS, with a double blocking treatment, in which the target was reinforced in compound with two previously reinforced CSs that were independently paired with the US. Note that the two blocking stimuli in the double blocking treatment each received the same amount of reinforcement as the single CS in the typical blocking treatment. Critically, a test of the target CS revealed more blocking (i.e., less responding to the target CS) in the single blocking condition relative to the double blocking condition. In other words, they observed a counteraction between two punctate comparator stimuli. This result is consistent with the ECH because compound training allowed the two blocking CSs to form a within-compound association, which caused them to act as effective second-order comparator stimuli for each other when the target CS was tested. This arrangement resulted in a reduction in both blocking CSs' potential to compete with the target CS.

Witnauer et al.'s (in press) observation is not completely congruent with other reports in the literature. For example, Kremer (1978) conducted an experiment that assessed the blocking potential of two excitatory CSs. His results indicated that a target stimulus can gain inhibitory potential after it has been reinforced in compound with two other CSs that both have strong excitatory associations with the US. Although subsequent studies have questioned whether Kremer observed genuine conditioned inhibition, these studies showed no evidence of a counteraction between two blocking CSs (also see Schachtman et al., 1985). However, these experiments did not include controls that allowed for the observation of blocking. Furthermore, they used a training regimen that included interspersed reinforcement of the two blocking CSs alone throughout the compound training phase. This sort of training could reduce the effective association between the two blocking stimuli, thereby attenuating their influence on each other.

3. Evidence from other preparations

The literature discussed above suggests that compound training can counteract the effectiveness of other treatments that typically reduce conditioned responding to an elementally trained CS. Most of these studies have been conducted by Miller and his colleagues using a conditioned suppression preparation with a footshock US. Some other instances of counteraction effects in other preparations are detailed below.

3.1. Conditioned taste aversion: latent inhibition and overshadowing

In a conditioned taste aversion (CTA) preparation with rats, Loy and Hall (2002) observed an apparent counteraction between latent inhibition and overshadowing. In their Experiment 5, half of the subjects received preexposure to an NaCl solution whereas the other half received no preexposure. Within each of those two conditions, half of the subjects then received pairings of sucrose and LiCl (together in a compound solution), whereas the other half received unpaired exposure to both solutions. After this treatment, the rate of consumption of NaCl was measured, with the assumption that a conditioned aversion to the taste of LiCl would generalize to NaCl. Strong conditioned responding was indexed by low consumption of the NaCl solution. Subjects that received either preexposure or compound training drank more NaCl than those that received neither treatment indicative of latent inhibition and overshadowing effects, respectively. Subjects that received both treatments drank more than those that received only preexposure of NaCl, but less than those that received compound training. In other words, they observed a counteraction effect when responding was compared to overshadowing alone, but not when compared to latent inhibition alone. This asymmetrical counteraction between latent inhibition and overshadowing is similar to that observed by Ishii (1999), except reversed in its asymmetry. When both treatments were administered, Ishii observed responding that was greater than his latent inhibition effect, whereas Loy and Hall observed responding that was greater than their overshadowing effect. There is, however, some doubt about the generality of Loy and Hall's result, which will be addressed later in this review.

3.2. Operant conditioning: US preexposure and overshadowing

Maier et al. (1987) observed a counteraction between US preexposure and overshadowing in a conditioned avoidance preparation. In their Experiment 1, subjects were divided into three groups that received escapable shock, inescapable shock, or no shock in wheel-turn boxes. All subjects were then given negative-reinforcement training in shuttleboxes in which the shock was escapable if the animals ran from one end of the chamber to the other. Furthermore, half of the subjects in each group experienced a signaling stimulus whenever they made a response (i.e., ran to the end of the shuttlebox). For the other half of the subjects, the response was unsignaled. Maier et al. found that preexposure to inescapable shock increased response latencies when subjects were subsequently given unsignaled escape training. This weakened conditioned responding is analogous to a US-preexposure effect in Pavlovian conditioning (e.g., Baker et al., 1981). Maier et al. also observed an effect that was analogous to overshadowing in Pavlovian conditioning. Subjects that received no preexposure to the US before signaled escape training showed slower conditioned responding. The signal apparently overshadowed the target response. However, subjects that received both of these response-degrading treatments exhibited responding that was faster than that observed with either treatment alone. Unlike the counteraction effects describe to this point, the ECH does not predict the result of Maier et al. because the training context, which the ECH assumes is a primary comparator stimulus, was different than the context of US preexposure. Thus, the training context–US association would not have been enhanced by US preexposure treatment unless there was strong generalization between contexts. Even so, the result is still noteworthy, in that it is analogous to that observed by Urushihara and Miller (2006) in a Pavlovian conditioning situation. Although the two phenomena might be driven by differentpsychological mechanisms, they do show that similar counteraction effects can be observed in very different situations.

3.3. Conditioned taste aversion: CS duration and overshadowing

Maier et al. (1987) and Loy and Hall (2002) both studied the interaction between compound training and a treatment that degrades the contingency between the CS and US. Westbrook et al. (1983) conducted a series of experiments using a CTA preparation that investigated the strength of conditioned aversions after compound or elemental training without degrading CS–US contingency. Similar to Urushihara et al. (2004), Westbrook et al. found a counteraction between CS duration and overshadowing. In their Experiment 2, they reinforced an odor with an illness-inducing injection of LiCl. The odor was either reinforced alone or reinforced in compound with a flavor. Additionally, subjects received either 2- or 15-min exposure to the odor or flavor-odor compound in a training context. After conditioning, rats suppressed consumption in the presence of the odor when it was reinforced elementally after a 2-min presentation or when it was reinforced in compound with the flavor after a 15-min session. Westbrook et al. interpreted their finding as evidence that longer exposure allowed the flavor and odor to form a strong within-compound association, which resulted in flavor-induced potentiation of the odor aversion. The ECH offers an alternative explanation of these results; a stronger association between the flavor and the context was formed due to the extended exposure period, which allowed the context and the flavor to both become effective comparator stimuli and thus counteract the other's ability to compete with the odor.

4. Recovery from a counteraction effect

According to the ECHs account of counteraction effects, the competing CSs modulate responding to the target at test. This assumption of the model leads to the prediction that posttraining extinction of either of the competing CSs should cause a reemer-gence of the effectiveness of the other to serve as a first-order comparator stimulus. In most of the aforementioned examples, posttraining extinction of the context should cause a reduction in responding due to a reemergence of overshadowing. Likewise, posttraining extinction of the overshadowing CS should cause a reduction in responding due to a reemergence of context-mediated cue competition. In many of the studies conducted by Miller and colleagues, the counteraction effects were investigated in experiments that involved such posttraining-extinction manipulations.

Urcelay and Miller (2006) studied the effect of posttraining extinction on the counteraction between overshadowing and degraded contingency in a sensory-preconditioning preparation.³ In their Experiment 3, extensive extinction of the context after compound training, which with elemental conditioning can cause an attenuation of the degraded contingency effect (e.g., Grahame et al., 1992), instead produced a reduction in conditioned responding to the target cue, presumably because of a reemergence of overshadowing. In a fourth experiment, massive extinction of the overshadowing cue that can produce a recovery from overshadowing (i.e., retrospective revaluation; e.g., Kaufman and Bolles, 1981) also appeared to produce a reemergence of the degraded contingency effect. In both situations, extinction of one of the two comparator stimuli allowed the other comparator stimulus to exert its full influence on the target and decrease responding accordingly.

The recovery from counteraction effects is not limited to Urcelay and Miller's (2006) studies of the degraded contingency effect. Other studies have shown a recovery from the counteraction between overshadowing and other context-mediated cue-competition effects including overtraining (Stout et al., 2003a) and trial massing (Stout et al., 2003b). Note that reversal of a counteraction effect by extinguishing one of the two comparator stimuli results in a reduction of behavioral control by the target cue. This is essentially a form of mediated extinction in that it is a decrease in conditioned responding as a result of extinguishing a companion cue. In accordance with predictions made by the ECH, one observes either retrospective revaluation or mediated extinction depending on the parameters and the underlying associative structure of the situation. However, there are other examples of retrospective revaluation and mediated extinction that do not appear to have this structure (e.g., Harris and Westbrook, 1998; Holland and Forbes, 1982).

5. Constraints on the counteraction effects

The counteractions described above are not necessarily parameter independent or pervasive. According to the ECH, there are a number of factors that determine whether a counteraction effect or, conversely, a summation effect will occur. Furthermore, there are almost certainly important factors that contribute to these effects that are outside the scope of the ECH.

5.1. The role of the competing CS

A conspicuous commonality in all of the aforementioned counteraction effects is the presence of an overshadowing or (in the case of Witnauer et al., in press) blocking effect. According to the ECH, compound training alone is not sufficient to produce a counteraction effect. A reliable cue-competition effect is necessary. This insight might help to explain some of the conflicting results encountered by researchers that have investigated the interactions between two treatments that can counteract each other. For example, Schnur (1971) found no counteraction between latent inhibition and overshadowing in a conditioned suppression preparation. In his studies, latent inhibition was apparent when a CS was preexposed prior to elemental training. This latent inhibition effect was not disrupted by the presence of another CS during training, even if that stimulus was presumed to be more salient based on the levels of conditioned responding after training. However, there was no direct evidence of overshadowing relative to an elementally trained control. This precludes the observation of a counteraction effect in two ways. Practically, a reliable overshadowing effect is necessary to show an attenuation of overshadowing. In terms of theory, a counteraction between latent inhibition and overshadowing is generally more likely to occur if both individual effects are strong. There are other studies of CS preexposure and overshadowing in the literature that are difficult to interpret for similar reasons (e.g., Carr, 1974; Schnur, 1975).

The observation of a counteraction between two treatments that would normally reduce conditioned responding is also constrained by the relative size of the individual response-degrading effects, as well as the statistical reliability of the effects. In Experiments 2 and 3 of Ishii's (1999) studies of CS preexposure and compound training in a conditioned suppression preparation, the author did observe reliable overshadowing and latent inhibition effects. But the sizes of the two effects were not equal. The overshadowing effect was consistently smaller than the latent inhibition effect. According to the ECH, the relative sizes of the effects need to be similar in order to observe a symmetrical counteraction effect. Otherwise, one of the competing stimuli (in this case the context) is still somewhat effective as a first-order comparator stimulus because the other competing stimulus (in this case the overshadowing CS) is not sufficiently strong to act as an effective second-order comparator stimulus.

The ECH anticipates a counteraction between effects that are a result of some form of cue competition. Although many of the individual phenomena described to this point can be interpreted as cue-competition effects, similar outcomes can be the result of different mechanisms. Hypothetically, in some situations overshadowing could occur because a loud auditory stimulus interferes with the perception of a relatively weak target auditory CS. In such a situation, the ECH would not anticipate a counteraction effect because perceptual masking is outside the scope of the model. In other situations, the model explicitly anticipates the absence of a counteraction effect. Urushihara and Miller (2007) specifically tested the limits of counteraction effects by varying their parameters to alter the underlying cause of the partial-reinforcement and CS-duration effects. As discussed previously, in their Experiment 1 they found that compound training was advantageous rather than detrimental if the CSs were of long durations, or if the trials were only partially reinforced. However, in their Experiment 2 they found no advantage of compound training when they increased the overall CS-exposure by a factor of 10, without altering the rate of reinforcement. According to the ECH, exposure to the CS in the training context builds the CS–context association (Link 2), but if it is assumed that extinction can occur during the early parts of an exceptionally long CS, then exposure will also weaken the CS–US association (Link 1). Thus, in situations in which there is a large amount of nonreinforced exposure to the CS, partial-reinforcement and CS-duration effects could be driven more by associative extinction than cue competition. In the latter situation, the ECH posits weak responding to the CS because the association between the CS and US is genuinely weak, and not simply because the expression of the association is muted by a comparator process. In such a situation, the model does not anticipate a robust counteraction effect.

5.2. Limitations on generalization to other preparations

There is some indication that certain counteraction effects do not generalize to all learning situations. Nakajima et al. (1999) conducted a series of experiments that investigated the interaction between CS preexposure and compound training in a CTA preparation. Although they did find latent inhibition of an elementally trained CS, they did not observe reliable overshadowing of a compound-trained CS. Even so, the combination of the two treatments produced responding weaker than that produced by either treatment alone. Thus, there clearly was a latent effect of compound training that became apparent when the target CS was preexposed prior to compound training.

It is possible that Nakajima et al.'s (1999) failure to observe a counteraction between latent inhibition and overshadowing indicates a fundamental difference between taste-aversion learning and conditioned fear. However, their result stands in contrast with Loy and Hall's (2002) experiment that used a CTA preparation. In order to resolve this discrepancy, Nakajima and Nagaishi (2005) later attempted to replicate Loy and Hall's counteraction between overshadowing and latent inhibition, but failed to do so. Instead of a counteraction between latent inhibition and overshadowing, Nakajima and Nagaishi observed summation of the two effects. The design of the replication was almost identical to the original, except the target CS (NaCl) was tested immediately after training, whereas Loy and Hall tested the overshadowing CS (sucrose) before testing the target. This appears to be an important distinction between the two experiments. Nakajima and Nagaishi suggested that testing the overshadowing CS before the target CS could have resulted in mediated extinction of the target CS, which would be more profound for subjects that only received compound training relative to subjects that received target CS preexposure prior to compound training. Because they had a design that was more appropriate for assessing the strength of responding to the target CS, Nakajima and Nagaishi concluded that their result was less ambiguous than that observed by Loy and Hall. If their conclusion is correct, it does further question whether the counteraction between latent inhibition and overshadowing will generalize to the CTA preparation.

Considering the evidence presented above, there is considerable doubt as to whether there could be a reliable counteraction between latent inhibition and overshadowing in a CTA preparation. This certainly could be viewed as evidence that CTA employs a fundamentally different sort of learning mechanism than most other forms of associative learning. However, this difference might be more related to the way that the context is treated in a typical CTA experiment. In CTA studies, training and testing typically occur in the animals' home cages. According to the ECH, the counteraction between latent inhibition and overshadowing is dependent on a strong association between the training context and the overshadowing CS. If the home cage is used as a training context, it will likely have very weak associations with all of the stimuli involved in training. In support of this view, there is evidence that context blocking (i.e., the US-preexposure effect) in a CTA preparation relies on the presence of injection cues during preexposure (de Brugada et al., 2004, 2005). That is, the home cages themselves do not appear to act as a competing cue because they are thoroughly extinguished between trials. According to the ECH, a counteraction effect would be more readily observed in a CTA preparation if training occurs outside of the home cages, or if there are some other cues present that might compete with the taste (e.g., injection cues). Furthermore, in the experiments by Westbrook et al. (1983) discussed earlier as evidence of a counteraction effect in a conditioned taste aversion, all training was conducted in experimental chambers in a room different from that housing the home cages.

Unfortunately, the observation that most CTA experiments use the home cages as the training context cannot fully resolve the conflicting reports in the literature. Ishii et al. (1999, Experiment 2) also observed summation of latent inhibition and overshadowing in a conditioned taste aversion preparation when using a training context other than the home cages. The rather consistent finding of summation of latent inhibition and overshadowing from different laboratories using CTA preparations certainly suggests that a counteraction effect is at least difficult to observe in the CTA preparation. However, Westbrook et al.'s (1983) observation of a counteraction between CS duration and overshadowing shows that other counteraction effects can occur in a CTA preparation.

6. Counteraction effects without overshadowing or blocking

Although the phenomena discussed to this point have largely involved counteractions between overshadowing and a context-mediated effect (e.g., degraded contingency) there are other counteraction effects that involve other treatments. For example, Urcelay and Miller (2008) observed a counteraction effect that involved conditioned inhibition instead of overshadowing. They examined the interaction between two different methods of training that can imbue a CS with inhibitory potential. One group of subjects received Pavlovian inhibitory training (A+/AX−). With this method, the punctate training excitor A drives the inhibitory potential of the target CS X (e.g., Lysle and Fowler, 1985; Hallam et al., 1990). A second group of subjects received explicitly unpaired inhibitory training (+/X−). With this method, there is no punctate training excitor, so the inhibitory potential of the target stimulus X is only driven by the context (e.g., Best et al., 1985). A third group received both types of training with trials interspersed (A+/+/AX−/X−). When the inhibitory potential of X was assessed in retardation and summation tests, Pavlovian training produced the strongest conditioned inhibitor. Explicitly unpaired training produced a weaker inhibitor, and the combination of the two treatments produced a CS that elicited no sign of inhibition. The group that received both types of inhibition training appeared to display a counteraction effect between the excitatory potential of A and the context, leaving little or no excitation to drive behavior indicative of inhibition. This result extends the generality of counteraction effects to situations that do not involve overshadowing or blocking. The ECH anticipates this counteraction effect in much the same way that the counteraction between overshadowing and degraded contingency is anticipated.

Witnauer and Miller (under review) observed two counteraction effects in a second-order conditioning situation. They obtained second-order conditioning by pairing a target stimulus (X) with a CS (A) that was reinforced alone on separate trials (A+/AX−), or by presenting X alone in a context that was made excitatory with unsignaled US presentations (+/X−). Although Urcelay and Miller (2008) observed conditioned inhibition with these treatments, Witnauer and Miller used significantly fewer trials. With a small number of trials, these treatments can result in excitatory responding to X rather than inhibition. With temporally spaced A+/AX− training, Witnauer and Miller observed reliably stronger responding to a target stimulus that was paired with an excitatory punctate CS, relative to a control group that lacked target stimulus presentations during training. That is, second-order conditioning was observed when the trials were spaced. Moreover, they observed strong responding after training consisting of unsignaled US presentations and target stimulus-alone presentations when trials were massed (+/X−), relative to an appropriate control group. Thus, when trials were massed the training context-mediated second-order conditioning to the target stimulus, and when trials were spaced the punctate CS A was able to mediate second-order conditioning. However, when trials were massed and the punctate CS was present during training, such that both the training context and A could mediate second-order conditioning, little responding to X was observed. In two follow-up experiments, Witnauer and Miller showed that pretraining reinforcement of the training context also reduced the potential of A to mediate second-order conditioning, and that posttraining extinction of the training context provided recovery from the counteraction effect. These results suggest that response-enhancing treatments can counteract each other in a manner similar to that observed in cue competition. Although the ECH as it was originally formulated fails to account for such positive mediation effects (e.g., second-order conditioning and sensory preconditioning), a recent mathematical implementation of the ECH (sometimes competing retrieval [SOCR]; Stout and Miller, 2007) accounts for such phenomena. SOCR, like the ECH, anticipates counteraction between select response-degrading treatments, but also predicts that response-enhancing treatments can counteract each other.

Wheeler and Miller (2007) observed two counteraction effects that are not anticipated by the ECH, but maintain some consistency with the spirit of the model. They investigated the potential interaction between degraded contingency and retroactive interference in a sensory-preconditioning preparation. When each of two stimuli are individually paired with a common outcome in separate phases of training (X+, followed by A+), impaired expression of the association between the first cue and the outcome is indicative of retroactive interference (e.g., Escobar et al, 2001). Wheeler and Miller examined whether unsignaled outcome exposures would interact with the retroactive inference effect. In their first experiment, subjects that received the retroactive interference treatment showed weak responding to X relative to a control group that experienced pairings of the target and the outcome followed by presentations of A alone (X+ followed by A−). Another group of subjects that received a degraded contingency treatment (+/X+, followed by +/A−) also showed weak responding relative to the control. However, subjects that received a combination of both treatments (+/X+, followed by +/A+) exhibited more responding than those that received retroactive interference or degraded contingency treatments alone. Similar to the result observed by Urcelay and Miller (2006), there was an apparent counteraction between retroactive interference and degraded contingency. In a subsequent experiment Wheeler and Miller observed an asymmetrical counteraction effect between trial massing and retroactive interference. In that experiment, trial massing and retroactive interference treatments both produced a deficit in responding to the target when administered individually to separate groups of subjects. Subjects that received both treatments combined exhibited greater responding than those that only received massed training trials, but no greater responding than those that only received retroactive interference treatment. Thus, the counteraction was not complete. Although that result was only partially analogous to the counteraction between trial massing and overshadowing observed by Urushihara et al. (2004), both experiments suggest that counteraction effects can be observed with an interference effect in which two cues are not directly associated with each other. Importantly, the training context was the same for both cues and thus was associated with each cue.

7. Conclusion

Considering the literature reviewed here, it is clear that the combination of two treatments that normally reduce conditioned responding to a target CS when they are administered separately can produce some unexpected results, namely less of an impairment of conditioned responding when they are administered together. Most of these counteraction effects are anticipated by Denniston et al.'s ECH (2001; for a mathematical implementation, see Stout and Miller, 2007), but it is important to note that the ECH is not the only model that can account for counteraction effects. Schmajuk et al.'s (1996; also see Schmajuk and Larrauri, 2006) model can at least account for the counteraction between latent inhibition and overshadowing by focusing on changes in novelty and the attention directed toward to the target CS. In addition, Courville et al. (2006) Bayesian theory of learning can account for the counteraction between latent inhibition and overshadowing by invoking increases in learning produced by the presence of an unexpected novel stimulus. We are not certain whether those models would predict all of the other counteraction effects presented here, but we suspect that they would fare well at least with some of the other contingency manipulations (e.g., US preexposure). However, it remains unclear whether either model could explain the recovery from counteraction effects, or the absence of a counteraction effect when the context is switched between CS-preexposure and compound training (Blaisdell et al., 1998).

Regardless of the theories used to explain these counteraction effects, it is clear that they are reliable, yet subject to some constraints. Miller et al. have consistently shown that overshadowing can counteract many different context-mediated response-degrading effects. Some of these counteraction effects might not easily generalize to other preparations, but there are some clear examples of counteraction effects that do not involve Pavlovian fear conditioning. Further research is underway to more thoroughly test the scope of these effects and the psychological mechanisms that are responsible.