Overshadowing and the Outcome-Alone Exposure Effect Counteract Each Other

Abstract

Three conditioned lever-press suppression experiments with rats investigated the interaction between overshadowing and outcome-alone exposure effects. Experiment 1 found in first-order conditioning that combined overshadowing and outcome-preexposure treatments attenuate the response deficit produced by either treatment alone. Experiments 2 and 3 investigated the interaction between overshadowing and outcome pre- and postexposure effects in sensory preconditioning, varying retention intervals to engage recency and primacy effects with respect to treatment order. Contrary to when a solitary cue is conditioned, responding to a cue conditioned in compound appeared positively correlated with the context’s associative status. These findings suggested that some of the basic laws of learning applicable to cues conditioned alone do not similarly apply to a component of a compound cue.

When a compound of two cues is followed by an outcome, behavioral control by each cue ordinarily is weaker than when either cue is trained alone (e.g., overshadowing; Pavlov, 1927). This class of phenomena, cue competition, has attracted a great deal of attention and has served as benchmarks for the development of various learning theories for more than three decades. Recently, several studies (e.g., Blaisdell, Bristol, Gunther, & Miller, 1998; Stout, Chang, & Miller, 2003; Urcelay & Miller, 2006; Urushihara, Stout, & Miller, 2004) reported a counterintuitive aspect of cue-competition phenomena. These studies investigated the generalization of the basic laws discovered in elemental conditioning situations to cue-competition situations and found that effects observed in elemental conditioning situations are often reversed in cue-competition situations. Blaisdell et al. (1998) found that pretraining exposure to a conditioned stimulus (CS), which creates a response deficit called the CS-preexposure effect (Lubow & Moore, 1959) when the CS by itself is later paired with an unconditioned stimulus (US), actually enhances responding if later training occurs in an overshadowing situation (see also Savastano, Arcediano, Stout, & Miller, 2003, for a replication and further elaboration). Stout et al. (2003) found that CS–US pairings with a short intertrial interval (ITI), which usually result in weaker responding to the target cue (the trial-massing effect; e.g., Gibbon, Baldock, Locurto, Gold, & Terrace, 1977; Jenkins, Barnes, & Barrera, 1981), actually produce stronger responding to the target cue after training in a cue-competition situation. Additionally, Urushihara, Stout, et al. (2004) determined that long CSs, which impair stimulus control when a single cue is paired with a US (the CS-duration effect; e.g., Coleman, Hemmes, & Brown, 1986; Pavlov, 1927), actually facilitate responding to the target cue if the target cue is trained in compound with a more salient cue and is tested alone. A recent study performed in our laboratory (Urcelay & Miller, 2006) found that US-alone presentations during the ITIs, which are known to decrease responding to a target cue (the degraded-contingency effect; Rescorla, 1968), actually enhanced responding to the target cue following training in a cue-competition situation.

The present report further focuses on laws that govern elemental conditioning situations that do not apply when cues are subject to a cue-competition effect. Specifically, we investigated the effect of US-preexposure prior to overshadowing treatment. The US-preexposure effect consists of retarded responding to a CS caused by US-alone exposures that precede CS–US pairings (e.g., Baker & Mackintosh, 1979; Mis & Moore, 1973; Randich, 1981; for a review, see Randich & LoLordo, 1979). Recent studies provide a good rationale for investigating the interaction between the US-preexposure effect and overshadowing. Most learning theories (e.g., Miller & Matzel, 1988; Rescorla & Wagner, 1972; Wagner, 1981) explain the US-preexposure effect as arising from some sort of competition between the contextual cue and the target CS. Specifically, repeated exposures to the US alone result in a formation of a strong association between the context and the US, which consequently blocks either acquisition or expression of the target cue–US association. These models also explain the trial-massing effect and the degraded-contingency effect in part by assuming that these treatments create a strong context–US association that competes with the CS–US association. Notably, both of these treatments have been found to counteract overshadowing (Stout et al., 2003; Urcelay & Miller, 2006, respectively). Given that the US-preexposure effect and these two effects (the trial-massing effect and degraded-contingency effect) seemingly arise from the same underlying mechanism, the US-preexposure effect may well also counteract overshadowing.

Investigation of the effect of US preexposure in a cue-competition situation is also of theoretical interest. Many learning theories have been developed to explain various forms of cue competition in learning including the US-preexposure effect and overshadowing (e.g., Miller & Matzel, 1988; Rescorla & Wagner, 1972; Wagner, 1981). These models make different predictions concerning the interaction between the US-preexposure effect and overshadowing. Some learning theories (e.g., Rescorla & Wagner, 1972; Wagner, 1981) predict that these two effects (US preexposure and overshadowing) should jointly cause weaker behavioral control by the target cue than either treatment alone (i.e., the effects should summate). This expectation follows from these models explaining cue-competition effects based on an assumption that a given outcome has a limited potential to support associative learning among multiple simultaneously presented cues. For example, the Rescorla–Wagner model hypothesizes that the magnitude of associative acquisition by a cue on a certain trial is a positive function of the difference between the asymptotic associative strength supportable by the US and total associative strength of all stimuli presented on that trial. Thus, when a highly salient cue is presented together with the target cue on a reinforced trial (i.e., an overshadowing procedure), the associative strength accrued to the target cue should be weaker than when the target cue is reinforced alone. Similarly, when the conditioning context has acquired a strong association with the US as a result of preexposure to the US within the context (i.e., a US-preexposure procedure), subsequent target cue–US pairings should result in weaker associative acquisition relative to a situation without the US-preexposure trials. In both overshadowing and US-preexposure treatment, the discrepancy between the asymptotic associative strength supportable by the US and the total associative strength of all stimuli presented at the time of target training becomes smaller. Therefore, this model predicts that when both of these manipulations are conducted with the same subjects, behavioral control by the target cue should be weaker than if either is conducted alone. This is because only the associative strength not assigned to the context during the US-preexposure treatment is available to be distributed across the overshadowing cue and the target cue during the overshadowing treatment.

The extended comparator hypothesis (Denniston, Savastano, & Miller, 2001; see Figure 1) generates a different prediction concerning the interaction between US-preexposure and overshadowing treatments. In the framework of the original comparator hypothesis (Miller & Matzel, 1988; see Figure 1, upper right), conditioned responding to the target cue is determined at the time of testing by the interaction of three associations, which are formed at the time of training according to principles of contiguity and salience. The first association is between the target CS and the US (Link 1), the second association is between the target CS and another stimulus present during training (i.e., comparator stimulus; Link 2), and the third association is between the comparator stimulus and the US (Link 3). At test, conditioned responding to the target CS is assumed to reflect a comparison of the US representations directly and indirectly activated by the target CS. Responding to the target CS is positively correlated with the strength of the directly activated US representation (Link 1) and negatively correlated to the strength of the indirectly activated US representation (the product of the strengths of Links 2 and 3). For example, when a target cue is paired with a US in compound with another cue of high salience (i.e., an overshadowing procedure), responding to the target cue should be weaker than when it is reinforced alone because Link 2 (the target cue–overshadowing cue association) and Link 3 (the overshadowing cue–US association) are both strong and they effectively down-modulate responding to the target cue. Similarly, responding to the target cue should be weaker when US-alone exposure is conducted before target training because Link 3 (the context–US association) is enhanced by the US preexposure and it down-modulates responding to the target cue. In the comparator framework, all cues possessing within-compound associations to the target CS act conjointly as comparator stimuli (Figure 1 depicts only one of what might be many first-order comparator stimuli).

Figure 1.

The extended comparator hypothesis (Denniston et al., 2001). CS = conditioned stimulus, US = unconditioned stimulus. Ovals represent stimulus representations, and rectangles represent the physical test stimulus and response. Conditioned responding to the target CS is determined by both the directly and the indirectly activated US representation at the time of testing: Direct activation of the US representation (Link 1) is positively correlated, and indirect activation of the US representation (the product of Links 2 and 3) is negatively correlated with the magnitude of conditioned responding. In the original comparator hypothesis (Miller & Matzel, 1988), responding to the target CS is down-modulated only by the absolute strength of Links 2 and 3, whereas in the extended comparator hypothesis, the effectiveness of each of these two comparator links is potentially influenced by its own comparator processes.

In the framework of the extended comparator hypothesis of Denniston et al. (2001), the basic assumption of the original comparator hypothesis is further expanded. That is, not only is Link 1 modulated by Links 2 and 3, but further comparator processes are presumed to modulate both Links 2 and 3 (see Figure 1). Thus, the effectiveness of the target CS–first-order comparator stimulus association (Link 2.1) is influenced by the product of the strength of the association between the target cue and any other potential comparator stimulus (second-order comparator stimuli for Link 2; Link 2.2) and the strength of the association between these other comparator stimuli and the first-order comparator stimulus (Link 2.3) through a comparator process similar to that which modulates Link 1 but applies to Link 2. Similarly, the effectiveness of the first-order comparator stimulus–US association (Link 3.1) is influenced by the product of the strength of the association between the first-order comparator stimulus and its own comparator stimuli (second-order comparator stimuli for Link 3; Link 3.2) and the strength of the association between the second-order comparator stimuli and the US (Link 3.3).

The important presupposition here is that each first-order comparator stimulus can simultaneously serve as a second-order comparator stimulus for the other first-order comparator stimuli. For example, when both the US-alone exposure and overshadowing procedures are conducted with the same subjects, the target cue should have two effective first-order comparator stimuli, the context and the overshadowing stimulus. At test, these two stimuli both act as the first-order comparator stimuli for the target cue, and these first-order comparator effects are summative. However, at the same time each of these two stimuli serves as a second-order comparator stimulus for the target cue; that is, they serve as first-order comparator stimuli for each other, resulting in a counteraction effect as long as the manipulations are similarly effective. More concretely, the US-preexposure effect, which arises from the context’s serving as a first-order comparator stimulus for the target cue, should be decreased by the second-order comparator effect of the overshadowing cue (see Figure 2, left). At the same time, the overshadowing effect, which arises from the overshadowing cue’s serving as a first-order comparator stimulus for the target cue, should be decreased by the second-order comparator effect of the context (see Figure 2, right). We defer more detailed theoretical discussion concerning the predictions of the extended comparator hypothesis until the General Discussion section, but the point to be emphasized here is that the extended comparator hypothesis explains cue-competition phenomena in a way other than hypothesizing a division of some limited resource among cues. Critically, it predicts a counteraction effect between the US-preexposure effect and overshadowing treatment with select parameters.

Figure 2.

An extended comparator hypothesis account of counteraction between overshadowing and the outcome-alone exposure effect. Left: The first-order comparator effect of the context. Right: The first-order comparator effect of the overshadowing stimulus. The two comparator processes on the left and the right work simultaneously at the time of testing. The thickness of each arrow represents the strength of the association. The higher-order comparator effect on Link 2 is omitted because it is negligible for the conjoint treatment condition.

In this report, we not only test the possibility of counteraction between overshadowing and the outcome-alone exposure effect, but also investigate its mechanism by testing a prediction of the extended comparator hypothesis. According to the extended comparator hypothesis, the counteraction effect between outcome-alone exposure treatment and overshadowing treatment should occur when the context and the overshadowing cue serve simultaneously as second-order comparator stimuli for each other.

This view of counteraction leads to the prediction that if the effectiveness of the context in competing with the overshadowing cue is impaired after training of the target cue, the counteraction effect between the two treatments should wane and be replaced by a simple overshadowing effect. As previously noted, two recent studies (Stout et al., 2003; Urcelay & Miller, 2006) have already reported that two basic phenomena—the trial-massing effect and the degraded-contingency effect, which are considered to arise from a process similar to that of the outcome-alone exposure effect—can counteract overshadowing. These two studies not only observed counteraction effects but also tested the above-mentioned prediction by the extended comparator hypothesis. Specifically, they found that the counteraction between overshadowing treatment and trial-massing treatment (Stout et al., 2003) or degraded-contingency treatment (Urcelay & Miller, 2006) could be converted into a simple overshadowing effect when the training context lost its potential to compete with the other cue through massive posttraining extinction of the context. In the present research, we also tested this prediction concerning a posttraining change in the effectiveness of the context as a competing stimulus, but we used a method other than that used in the previous studies.

A recent study by Urushihara, Wheeler, and Miller (2004) suggests an alternative method to control the effectiveness of the training context other than posttraining extinction of it. In a first-order conditioning situation, US-alone exposure is known to be effective when it is conducted prior to conditioning (i.e., forward blocking by the context) but far less so when conducted following conditioning (i.e., backward blocking by the context; e.g., Miller, Hallam, & Grahame, 1990). However, Urushihara, Wheeler, et al. recently found that, in a sensory-preconditioning preparation, interference caused by outcome postexposure is actually stronger than that by outcome preexposure, which is indicative of a recency effect. That is, when the interfering cue (context) was trained closer to the time of testing, a greater effect of this treatment was observed. Additionally, they further found that this recency effect in an outcome-alone exposure paradigm shifted to a primacy effect (i.e., stronger blocking by outcome preexposure than that by outcome postexposure) when testing was conducted after a long retention interval (i.e., a recency-to-primacy shift with increasing retention interval). These results suggest that the effect of outcome-alone exposure is governed by the recency-to-primacy principle (e.g., Konorski & Szwejkowska, 1952), which might be viewed as being caused by changes of the temporal context. Taking into consideration the results of Urushihara, Wheeler, et al., one might well expect the interaction between overshadowing and outcome-alone exposure effects in a sensory-preconditioning paradigm to be affected not only by changes in the order of outcome-alone exposures and cue–outcome pairings but by changes in the duration of the retention interval between training and testing. Notably, using the recency-to-primacy shift to control the effectiveness of the context is possible in outcome pre- and postexposure paradigms, whereas it is not in cue-competition procedures that involve a single phase of training such as the trial-massing or the degraded-contingency procedure, because the outcome pre- and postexposure paradigms engender phasic training.

In the present series of experiments, the interaction between overshadowing and US (outcome)-alone exposure effects was investigated in a conditioned suppression preparation with rats. In Experiment 1, the interaction of overshadowing and US-preexposure was examined in a first-order conditioning situation. Because US-postexposure treatment has low efficacy in a first-order conditioning situation (e.g., Miller, Hallam, et al., 1990), the US-postexposure treatment was not investigated in Experiment 1. In Experiment 2, the interaction between overshadowing and outcome-postexposure effect as well as outcome-preexposure effect was examined in a sensory-preconditioning situation. Experiment 3 was a replication of Experiment 2 but with a 3-week retention interval prior to testing. Our goals were to test the counterintuitive prediction of the extended comparator hypothesis that US (outcome)-alone exposure can counteract overshadowing treatment and to probe the role of the context in this interaction. Whereas past studies have manipulated the associative status of the context through posttraining associative deflation (i.e., extinction) and inflation, here we used the recency-to-primacy shift (as a function of retention interval) to manipulate the status of the context. Prior studies of direct stimulus control have demonstrated the recency-to-primacy shift (e.g., Stout, Amundson, & Miller, in press; Urushihara, Wheeler, et al., 2004; Wheeler, Stout, & Miller, 2004); here we used the recency-to-primacy shift to assay the comparator roles of the training context before and after target cue conditioning.

Experiment 1

The purpose of Experiment 1 was to investigate the interaction between overshadowing treatment and US-preexposure in first-order conditioning. Unlike the following two experiments, we focused only on the US-preexposure effect and not the US-postexposure effect. This is because, as noted in the introduction, the US-postexposure procedure is generally ineffective in first-order conditioning situations. Experiment 1 consisted of four groups. Two preexposure (Pre) groups (Pre-Acq and Pre-OV) experienced US-alone exposures prior to conditioning, whereas two control (Ctrl) groups (Ctrl-Acq and Ctrl-OV) experienced only equivalent context exposure. Orthogonally, two acquisition (Acq) groups (Pre-Acq and Ctrl-Acq) received elemental conditioning to the target cue, whereas two overshadowing (OV) groups (Pre-OV and Ctrl-OV) received conditioning to the target cue in compound with another more salient cue (i.e., an overshadowing procedure; see Table 1). We anticipated that the Pre-Acq and Ctrl-OV groups would both exhibit weaker responding than the Ctrl-Acq group. Our interest was whether these two effects would summate or counteract each other when the two procedures were combined in the Pre-OV group.

Table 1.

Design Summary of Experiment 1

Groups	Conditioning (context train)	Test (context test)
Pre-Acq	36 US → 4 X-US	4X
Ctrl-Acq	- - - → 4 X-US	4X
Pre-OV	36 US → 4 AX-US	4X
Ctrl-OV	- - - → 4 AX-US	4X

Note. Pre = US pretraining exposure, Ctrl = control, Acq = acquisition control, OV = overshadowing, X = target cue, A = overshadowing cue, US = footshock unconditioned stimulus, “-” = sequential pairing of stimuli, “- - -” = no treatment. Numbers refer to the number of each trial type. Context train and context test differed in illumination of the chamber, the odors present, the material constituting the floor, and the presence of an operant lever.

Method

Subjects

Twenty-four male (243–299 g) and 24 female (180–208 g) experimentally naïve Sprague–Dawley-descended rats bred in our colony served as subjects. The animals were randomly assigned to one of four groups (ns = 12), Pre-Acq, Ctrl-Acq, Pre-OV, and Ctrl-OV, counterbalanced for sex. The animals were individually housed in wire mesh cages in a vivarium maintained on a 16-hr light/8-hr dark cycle. Experimental manipulations occurred approximately midway through the light portion of the cycle. A progressive water deprivation schedule was imposed over four days prior to the beginning of the experiment until water availability was limited to 30 min per day. All subjects were handled for 30 s three times per week from weaning until the initiation of the study.

Apparatus

The apparatus consisted of 12 operant chambers each measuring 30 × 30 × 27 cm (l × w × h). All chambers had clear Plexiglas ceilings and sidewalls and metal front and back walls. On one metal wall of each chamber, there was a 3.5-cm wide operant lever on the left side, 4 cm above the floor, and a niche (2.5 × 4.5 × 4 cm) on the right side, the bottom of which was 2 cm above the floor, where a drop (0.04 μl) of distilled water could be presented by a solenoid valve. The floor was constructed of 0.3-cm diameter rods, spaced 1.3 cm center to center, and connected by NE-2 neon bulbs that allowed a constant-current footshock to be delivered by means of a high voltage AC circuit in series with a 1.0-MΩ resistor. Each chamber was housed in its own environmental isolation chest, which could be dimly illuminated by a 1.12-W incandescent house-light mounted on the front wall of the experimental chamber. Ventilation fans in each enclosure provided a constant 76-dB (C-scale) background noise. A 60-W incandescent bulb was mounted on the back wall of each environmental chest 26 cm from the center of the floor of the conditioning chamber. This bulb could be flashed (0.25 s on/0.25 s off) to serve as a visual stimulus and was used in Experiments 2 and 3, but not in Experiment 1. Three 45-Ω speakers mounted on the interior right, left, and back sides of each environmental chest were used to deliver a complex tone (3000 and 3200 Hz, 18 dB [C-scale] above the background), a click train (6/s, 5 dB [C-scale] above the background), and a white noise (8 dB [C-scale] above the background), respectively. In Experiment 1, the target cue (CS X) was a 30-s click train and the overshadowing cue (CS A) was a 30-s complex tone. The white noise was used to make delivery of water more conspicuous, thereby facilitating shaping of lever pressing. A 0.5-s white noise accompanied delivery of each drop of water reinforcement. The US was a 0.5-s, 0.5-mA footshock.

Contextual Cues

From the 12 chambers noted above, two distinct contexts were created that differed in visual, tactile, and odor cues. We used different contexts for conditioning and the other phases to minimize any difference in baseline responding among groups at the time of testing that might have been caused by preexposure to unsignaled aversive USs causing strong contextual fear to the conditioning context in the Pre-Acq and Pre-OV groups. In one context (Context Test), the houselight was illuminated, the operant lever protruded through the wall, and the grid floor was covered by a clear Plexiglas plate. In the other context (Context Train), the houselight was turned off; the operant lever was retracted; and an odor stimulus, which was produced by two drops of 98% methyl salicylate (a mint odor) on the top surface of a wooden cube placed inside the environmental isolation chest but outside of the experimental chamber, was presented. Baseline training, baseline recovery, and testing were conducted in Context Test, whereas all Pavlovian conditioning manipulations were conducted in Context Train. In order to make the contextual difference more conspicuous, the chamber used as Context Train for each subject was different from that used as Context Test for that subject (e.g., for Subject 1, Box 1 was used as Context Test and Box 12 was used as Context Train).

Procedure

Baseline training

On Days 1–7, lever-press shaping was conducted during daily 60-min sessions in Context Test. On Days 1 and 2, a fixed-time 120-s schedule of noncontingent water delivery operated concurrently with continuous reinforcement for lever pressing. Each water delivery was accompanied by a 0.5-s white noise in these and all of the following sessions. On Day 3, subjects were trained on the continuous reinforcement schedule alone. Subjects that made less than 50 responses on this day experienced a hand-shaping session later in the same day. On Days 4–7, subjects were trained under a variable-interval (VI) 20-s schedule. On Day 7, all subjects received two presentations of each CS during the 60-min session in order to reduce unconditioned suppression to the auditory cues. The order of presentation of CSs was counterbalanced between subjects within groups.

Conditioning

On Days 8–11, conditioning sessions were conducted during daily 90-min sessions in Context Train. Subjects in Groups Pre-Acq and Pre-OV received 10 daily presentations of the footshock US with an average ITI of 9 min (range = 6–12 min). The last four US presentations on Day 11 were preceded by either 30 s of CS X (the Pre-Acq group) or 30 s of the AX compound (the Pre-OV group). All subjects in the Ctrl-Acq and Ctrl-OV groups received the same manipulations as the corresponding Pre groups except that unsignaled US presentations on Days 8–11 were eliminated (i.e., they received only the four CS–US pairings on Day 11). During the CS–US pairings, the 0.5-s US was present during the last 0.5 s of the 30-s CS presentation.

Baseline recovery

On Days 12–15, baseline recovery training on the VI 20-s schedule was conducted in Context Test. During this period, all subjects received a 1-hr daily session. No nominal stimulus was presented during each session.

Test X

On Day 16, suppression of baseline responding during presentation of CS X was assessed in all groups in Context Test. Each subject received four nonreinforced 30-s presentations of CS X during a 30-min session initiated at 6, 13, 22, and 29 min into the session. The response rates (per minute) during the 60-s periods preceding each CS exposure (pre-CS score) and that during the 30-s CS exposure (CS score) were recorded.

Test A

On Day 17, suppression of baseline responding during presentation of CS A was assessed in all groups in Context Test in the same manner as Day 16 except for the stimulus tested. Although the test of A was not a central focus of the present experiment, it was expected to provide data complementary to the test of X.

A suppression ratio (Annau & Kamin, 1961) for each subject was calculated by the formula P/(P + Q) for Days 16 and 17, where P is the number of lever presses during the four 30-s CS presentations and Q is one half the number of lever presses during the four 60-s pre-CS periods. A pre-CS period twice as long as the CS presentation was adopted in order to reduce the variability in baseline response rate. A suppression ratio of .50 indicates no suppression, indicative of no conditioned responding, and that of .00 indicates complete suppression, indicative of strong conditioned responding.

Results and Discussion

Mean suppression ratios calculated with pre-CS and CS scores from Day 16 are depicted in Figure 3 (left). The Ctrl-Acq group suppressed more than the Pre-Acq and Ctrl-OV groups, suggesting a US-preexposure effect and an overshadowing effect. More important, suppression in the Pre-OV group, which received both the US-preexposure and overshadowing treatments, was stronger than those in the Pre-Acq and Ctrl-OV groups (which received only one of these two treatments) and almost comparable to that in the Ctrl-Acq group. This suggests that US-preexposure and overshadowing counteracted each other, which is consistent with the prediction of the extended comparator hypothesis, but contrary to those of traditional acquisition-focused models of learning (e.g., Rescorla & Wagner, 1972). These conclusions were confirmed by the following statistical analyses.

Figure 3.

Mean suppression ratio to the target cue (X; left) and the overshadowing cue (A; right) in Experiment 1. Strong fear to the CS is denoted by a low suppression ratio. During training, X was paired with a footshock US in compound with an overshadowing cue (A) in the Pre-OV and Ctrl-OV groups, whereas X was paired with the US without any other stimulus present in the Pre-Acq and Ctrl-Acq groups. US-alone exposures were administered before the X–S or AX–S pairings in the Pre-Acq and Pre-OV groups, whereas the Ctrl-Acq and Ctrl-OV groups received no US-alone exposure. Error bars represent the standard error of the mean.

In order to assess any potential differences in fear of the test context, pre-CS lever presses were first analyzed. Mean pre-CS lever presses (per minute) pooled across all test trials on Day 16 were 18.1, 19.1, 17.3, and 15.0 for Groups Ctrl-Acq, Ctrl-OV, Pre-Acq, and Pre-OV, respectively. A 2 (US exposure: Ctrl vs. Pre) × 2 (training condition: Acq vs. OV) analysis of variance (ANOVA) revealed no main effect or interaction, Fs(1, 44) = 1.72. A similar 2 × 2 ANOVA conducted on the suppression ratios revealed main effects of US exposure, F(1, 44) = 16.18, p < .01, and training condition, F(1, 44) = 9.86, p < .01, and an interaction, F(1, 44) = 91.06, p < .01. Planned comparisons using the error term of the latter ANOVA revealed that the suppression by the Pre-Acq group was smaller than that by the Ctrl-Acq group, F(1, 44) = 92.01, p < .01, indicative of the conventional US-preexposure effect, and that the suppression by the Ctrl-OV group was smaller than that by the Ctrl-Acq group, F(1, 44) = 20.49, p < .01, indicative of the conventional overshadowing effect. Further planned comparisons revealed that the suppression in the Pre-OV group was greater than that in the Ctrl-OV and Pre-Acq groups, F(1, 44) = 15.23, and F(1, 44) = 80.42, ps < .01, indicating that the US-preexposure effect and the overshadowing effect counteracted each other.

The mean suppression ratio in response to CS A on Day 17 of each group is depicted in Figure 3 (right). The Pre-OV group showed weaker suppression than the Ctrl-OV group, suggesting a US-preexposure effect to the overshadowing stimulus (A). However, despite the fact that the Ctrl-Acq and Pre-Acq groups received no A–US pairings, these two groups differed in the same direction as that between the two overshadowing groups; however, the magnitude was smaller. This tendency makes the results of CS A testing inconclusive. One possible explanation for the difference between the Ctrl-Acq and Pre-Acq groups is stimulus generalization from CS X to CS A; the difference in suppression to CS A between these two groups is in the same direction as that to CS X. However, the difference between the Ctrl-OV and the Pre-OV groups cannot be explained with the notion of stimulus generalization because the difference in suppression to CS A between these two groups is in the opposite direction to the difference in suppression to CS X. The suppression data to CS A was statistically analyzed in the same manner as the CS X data. Mean pre-CS lever presses (per minute) pooled across all four test trials on Day 17 were 19.7, 20.5, 19.7, and 14.3 for Groups Ctrl-Acq, Ctrl-OV, Pre-Acq, and Pre-OV, respectively. A 2 (US exposure) × 2 (training condition) ANOVA revealed no significant main effect or interaction, Fs(1, 44) = 3.16. A similar 2 × 2 ANOVA conducted on the suppression ratio of each group revealed main effects of US exposure, F(1, 44) = 16.41, p < .01, and training condition, F(1, 44) = 8.24, p < .01. The interaction was not significant (F < 1).

The results of Experiment 1 demonstrate that the US-preexposure effect can be reversed by overshadowing treatment. Similarly, the US-alone exposure procedure, which retarded responding to CS X in the Pre-Acq group compared to the Ctrl-Acq group, enhanced responding in the Pre-OV group relative to the Ctrl-OV group. As previously noted, these results are contrary to the prediction of most acquisition-focused models such as the Rescorla–Wagner model (Rescorla & Wagner, 1972). These acquisition-focused models explain both the US-preexposure effect and overshadowing as competition for acquisition of a limited resource of the US between the target CS and the overshadowing cue or the context. Thus, when two procedures are administered conjointly, the two effects should summate. The present results pose a serious challenge to acquisition-focused models, which explain cue-competition phenomena as a division of a resource such as total associative strength among multiple cues.

In contrast, the extended comparator hypothesis can explain the results of Experiment 1. In the framework of this model, both the context and the overshadowing cue work as effective first-order comparator stimuli of the target cue when the US-alone exposure and the overshadowing procedure are both administered. The representations of these two first-order comparator stimuli are simultaneously activated at test, and their deleterious effects upon stimulus control are expected to summate. However, at the same time, each stimulus works as a second-order comparator stimulus for the target cue, that is, a first-order comparator stimulus for the other first-order comparator stimulus thereby down-modulating the effectiveness of the other first-order comparator stimulus. As a result, the total effectiveness of the two first-order comparator stimuli in down-modulating responding to the target cue can become weaker when two procedures are combined than when either procedure is employed alone. It is important to note, however, that this explanation of the current results by the extended comparator hypothesis is parameter dependent. We will defer further discussion on this point until the General Discussion section.

Recent studies have reported that phenomena which are often viewed as cue competition between the target cue and the conditioning context, such as the trial-massing effect (Stout et al., 2003) and the degraded-contingency effect (Urcelay & Miller, 2006), counteract overshadowing. The findings of Experiment 1 can be taken as a variant on these previous findings. The extended comparator hypothesis predicts that massed presentations of the US (the trial-massing effect) and extra US-alone exposures interspersed among (the degraded-contingency effect) or prior to (the US-preexposure effect) target training result in formation of a strong context–US association, which consequently causes counteraction in overshadowing situations. One may well point out that, in this framework, US-alone exposures following the target training (i.e., US-postexposure) should have the same effect as US-preexposure. Both of these two manipulations are expected to contribute to a strong context–US association by the time of testing. However, preliminary experiments in our laboratory as well as prior studies (e.g., Miller, Hallam, & Grahame, 1990) have shown that the US-postexposure effect is ineffective in the first-order conditioning situation. Savastano and Miller (2003) suggested that, once a stimulus has gained the potential to control behavior (i.e., has become biologically significant), it is difficult to degrade that potential of the stimulus to control behavior through indirect means, such as manipulating its competing cue. We investigated in Experiment 1 the effect of US-preexposure and not of US-postexposure because of this difficulty in obtaining the US-postexposure effect in first-order conditioning situations. However, this problem arising from the biological significance of a stimulus can be circumvented by embedding the experiment into a sensory-preconditioning paradigm. In sensory-preconditioning situations, stimuli can become associated without their becoming biologically significant. Thus, in the two following experiments, the interaction between overshadowing and outcome postexposure as well as outcome preexposure was investigated using a sensory-preconditioning paradigm.

Experiment 2

The purpose of Experiment 2 was to assess the generality of the finding of Experiment 1 and illuminate the processes underlying it. Note that Experiment 2 was embedded in a sensory-preconditioning paradigm rather than conducted in first-order conditioning. The primary reason for the use of a sensory-preconditioning procedure was to allow us to investigate the outcome-postexposure effect as well as the outcome-preexposure effect. The associative inflation of a competing cue following target training (e.g., backward blocking), including instances in which the competing stimulus is the training context (e.g., the US-postexposure effect), has generally proven ineffective in first-order conditioning situations (e.g., Miller, Hallam, & Grahame, 1990). This ineffectiveness has been viewed as arising from the target cue acquiring control of behavior during the training phase (which is not the case when the competing cue is inflated prior to target training). Specifically, once a stimulus obtained a potential to control behavior, it seemingly becomes difficult to degrade that potential by indirect means such as manipulating the status of competing stimuli (Denniston, Miller, & Matute, 1996; Savastano & Miller, 2003). Thus, in order to investigate the effect of the US postexposure, it is necessary to circumvent the target cue from initially coming to control behavior. One way to do this is to embed the target training phase in a sensory-preconditioning situation in which all stimuli are kept biologically nonsignificant until the end of target training.

In Experiment 2, we further investigated the mechanism of the counteraction effect that we found in Experiment 1. As previously mentioned, the counteraction effect between US-preexposure treatment and overshadowing treatment is explicable in terms of the extended comparator hypothesis assuming that the context and the overshadowing cue each serve simultaneously as a second-order comparator stimulus for the other. This account leads to another testable prediction. If the effectiveness of the context as a comparator stimulus for the overshadowing cue is decreased (i.e., extinction of the training context) following training of the target cue, the counteraction effect between the US-preexposure effect and overshadowing should be reduced and replaced by a simple overshadowing effect. Prior studies have reported analogous effects. These studies found that the counteraction between overshadowing treatment and trial massing (Stout et al., 2003) and between overshadowing treatment and degraded-contingency treatment (Urcelay & Miller, 2006) could be converted into a conventional overshadowing effect by massive posttraining extinction of the training context, which is consistent with predictions of the extended comparator hypothesis. In Experiments 2 and 3, we tested this prediction concerning a posttraining change in the effectiveness of the context as a competing stimulus of the target cue with a method other than the posttraining deflation.

Urushihara, Wheeler, et al. (2004), using a sensory-preconditioning preparation, found that the effect of outcome-alone exposure obeys a recency-to-primacy principle. That is, when testing of the target cue was conducted immediately after training, not only was outcome-posttraining exposure effective in attenuating responding to the target cue, outcome-posttraining exposure actually caused stronger blocking by the context than did outcome-pretraining exposure (i.e., a recency effect of outcome-alone exposure), which is contrary to what is observed in first-order conditioning situations. In contrast, this superiority of the outcome-postexposure effect was replaced by a superiority of the outcome-preexposure effect when testing was conducted after a long retention interval (i.e., a recency-to-primacy shift was observed with an increasing retention interval). This finding suggests that one can control the effectiveness of outcome-alone exposure by manipulating either the order of outcome-alone exposures and cue–outcome pairings or the retention interval between training and testing. In Experiments 2 and 3, we made use of the findings of Urushihara, Wheeler, et al. to investigate the role of the training context in the counteraction effect observed in Experiment 1.

The use of a recency-to-primacy shift of outcome-alone exposure effects, rather than posttraining associative deflation (i.e., extinction) of the training context that was employed in previous studies (e.g., Stout et al., 2003; Urcelay & Miller, 2006), not only assesses generality across different means of varying the associative status of the context, but also has a theoretical rationale. As pointed out in Urushihara, Wheeler, et al. (2004), no contemporary associative learning theory can explain the recency-to-primacy shift seen in the outcome-alone exposure paradigm. Most associative learning theories assume that individual events during training are not encoded, but instead are transformed into summary statistics such as associative strengths between stimuli. Thus, it is difficult for these models to explain a phenomenon in which representations of different treatment events (e.g., outcome-alone exposure and cue–outcome pairings) appear to be differentially retrieved at the time of target cue testing and conjointly influence stimulus control. The recency-to-primacy shift observed in an outcome-alone exposure paradigm, reported by Urushihara, Wheeler, et al., illustrated this limitation of conventional associative learning theories. In the following experiments, we sought to expand the findings of Urushihara, Wheeler, et al. to a cue-competition situation. Determining whether the counteraction between overshadowing and the outcome-alone exposure effect is also influenced by a recency-to-primacy shift would provide information concerning whether manipulating the order of training and the duration of retention interval influence the second-order comparator value of stimuli as it has been found to do for the first-order comparator value of stimuli (Urushihara, Wheeler, et al.) and direct excitatory value (e.g., spontaneous recovery from extinction; Pavlov, 1927).

Experiments 2 and 3 investigated the counteraction between overshadowing and the outcome-alone exposure effect in a sensory-preconditioning preparation with a short and long retention interval, respectively. The design of Experiment 2 is summarized in Table 2. Six groups of subjects received different treatments in a 3 (outcome exposure: outcome preexposure [Pre], outcome postexposure [Post], or no exposure [Ctrl]) × 2 (training condition: target CS alone [Acq] or compound CS [OV]) factorial design. The Acq groups received the target cue–surrogate outcome pairings (i.e., X–S), whereas the OV groups received pairings of a compound cue, consisting of the overshadowing stimulus and X, with S (i.e., AX–S). Furthermore, the Pre and Post groups received S-alone exposures preceding the cue–outcome pairings or S-alone exposures following the cue–outcome pairings, respectively. The Ctrl groups received no S-alone exposures. The subjects in the two Ctrl groups were divided into two subgroups, according to whether they received cue–outcome pairings in Phase 1 or 2. This was done in order to determine if the 2-day difference in retention interval between the Pre and Post conditions had any effect. If no differences were detected across these subgroups, they were to be pooled and we could surmise that the 2-day difference in retention interval between the Pre and Post groups was inconsequential. All subjects then received first-order conditioning of the surrogate outcome (S–US). Finally, conditioned suppression to X was tested after two baseline recovery sessions.

Table 2.

Design Summary of Experiments 2 and 3

Groups	Phase 1	Phase 2	Phase 3	Test Exp 2	Interval	Test Exp 3
Pre-Acq	24 S	4 X-S	4 S-US	2 X	21 days	2 X
Post-Acq	4 X-S	24 S	4 S-US	2 X	21 days	2 X
Ctrl-Acq	- - -	4 X-S	4 S-US	2 X	21 days	2 X
	4 X-S	- - -
Pre-OV	24 S	4 AX-S	4 S-US	2 X	21 days	2 X
Post-OV	4 AX-S	24 S	4 S-US	2 X	21 days	2 X
Ctrl-OV	- - -	4 AX-S	4 S-US	2 X	21 days	2 X
	4 AX-S	- - -

Note. Pre = outcome pretraining exposure, Post = outcome posttraining exposure, Ctrl = control, Acq = acquisition control, OV = overshadowing, X = target cue, A = overshadowing cue, S = surrogate outcome, US = footshock unconditioned stimulus, “-” = sequential pairing of stimuli, “- - -” = no treatment. Numbers refer to the number of each trial type. Control groups were composed of two subgroups that received X-S or AX-S trials either in Phase 1 or Phase 2. In Experiment 2, baseline recovery was conducted immediately after Phase 3 treatment and followed by testing, whereas in Experiment 3, baseline recovery and testing were delayed 21 days. During this retention interval, subjects remained in their home cages without any experimental manipulation.

In order to achieve recency and primacy effects in outcome-alone exposure paradigms, the designs of Experiments 2 and 3 were embedded within a sensory-preconditioning preparation with the procedural parameters modeled on those used by Urushihara, Wheeler, et al. (2004) rather than those used in Experiment 1. Note that this resulted in several differences in major experimental procedures as well as minor parameters between Experiments 1 and 2. First, the change of physical context between conditioning and testing in Experiment 1 was not used in Experiment 2. The change in physical context in Experiment 1 was necessary because of the many unsignaled shocks administered. In contrast, in Experiment 2 the analogous multiple exposures to the outcome (S) during the outcome-alone exposure was not expected to cause a substantial difference in contextual fear among groups; the number of presentations of the actual US (footshock) was identical among groups in Experiment 2. A second change from Experiment 1 was that the outcome-alone and the CS-outcome trials in Experiment 2 were separated into two different phases as was done by Urushihara, Wheeler, et al. in order to increase the likelihood of our obtaining a recency-to-primacy shift. Other minor changes in parameters including the number of CS-outcome trials, the number of outcome-alone exposures, the ITI, and the session length in the various training phases were also changed to be in accord with Urushihara, Wheeler, et al. We should emphasize here that the parameters used in Urushihara, Wheeler, et al. were selected with the intent of maximizing primacy and recency effects. That is, with these parameters we expected at best a weak outcome-preexposure effect, which would allow us to clearly evaluate any recency effect that occurred. As a result, contrary to the results in Experiment 1, we expected little outcome-preexposure effect but a strong outcome-postexposure effect with a test soon after training. This does not mean that the outcome-preexposure effect cannot be obtained in a sensory-preconditioning paradigm; theoretically, it is possible to obtain a considerable outcome-preexposure effect with appropriate parameters (such as more outcome-alone preexposure trials).

Based on Urushihara, Wheeler, et al. (2004), outcome postexposure was expected to have a stronger deleterious effect on suppression to the target cue than did outcome preexposure when the target cue was trained by itself (i.e., among the Acq groups) provided testing was conducted soon after training, as was the case in Experiment 2. Thus, conditioned suppression in the Post-Acq group was expected to be weaker than in the Pre-Acq and Ctrl-Acq groups. Our main interest was in the behavior of the three OV groups relative to their corresponding Acq groups. According to the extended comparator hypothesis, the counteraction between the outcome-alone exposure effect and overshadowing should occur only when the context works as an effective comparator stimulus for overshadowing stimulus. Thus, although the extended comparator hypothesis cannot explain why the recency effect in an outcome-alone exposure paradigm occurs, given that the recency effect does occur, it predicts that a counteraction effect will occur more strongly in the Post condition and an ordinary overshadowing effect is more apt to be observed in the Pre condition.

Method

Subjects and Apparatus

Thirty-six male (219–337 g) and 36 female (172–214 g) experimentally naïve Sprague–Dawley-descended rats bred in our colony served as subjects. Housing, deprivation, and handling conditions were the same as in Experiment 1. The animals were randomly assigned to one of six groups (ns = 12), Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV, counterbalanced for sex. The apparatus was the same as that described in Experiment 1. As in Experiment 1, the tone served as CS A and the clicks as CS X. A 5-s flashing (0.25 s on/0.25 s off) light provided by a 60-W (at 120 VAC, but driven at 50 VAC) incandescent bulb mounted on the back wall of each environmental chest was used as the surrogate outcome, S. Unlike in Experiment 1, all experimental manipulations were conducted in the same context, in which the houselight was turned on, the operant lever was present, the grid floor was not covered by a Plexiglas plate, and no odor stimulus was present.

Procedure

Baseline training

Baseline training was conducted in the same manner as Experiment 1, except that the VI 20-s training was reduced to 2 days (because the subjects had opportunities to learn to press the lever throughout all phases) and the brief pretreatment exposures to X and A were conducted on Day 5. The VI 20-s schedule prevailed throughout the remainder of the experiment.

Phase 1

On Days 6 and 7, Phase 1 treatment was conducted during 1-hr sessions. Subjects in the Pre-Acq and Pre-OV groups received 12 daily presentations of S alone with a mean ITI of 5 min (range = 3.5–7.5 min). All subjects in the Post-Acq group and half of the subjects in the Ctrl-Acq group received two daily pairings of X and S with an ITI of 30 min. All subjects in the Post-OV group and half of the subjects in the Ctrl-OV group received two daily pairings of the AX-compound stimulus and S (AX–S) in the same manner. During these pairings, the 5-s S stimulus coterminated with the 30-s CS. The remaining half of the subjects in the Ctrl-Acq and Ctrl-OV groups received no presentation of any nominal stimulus in these sessions (see Table 2).

Phase 2

On Days 8 and 9, Phase 2 treatment was conducted during 1-hr sessions. All subjects in the Pre-Acq group and the subjects in the Ctrl-Acq group that did not receive X–S pairings in Phase 1 received two daily X–S pairings in the same manner as did subjects in the Post-Acq group during Phase 1. All subjects in the Pre-OV group and the subjects in the Ctrl-OV group that did not receive AX–S pairings in Phase 1 received two daily AX–S pairings in the same manner as did subjects in the Post-OV group during Phase 1. Subjects in the Post-Acq and Post-OV groups received 12 daily presentations of S alone in the same manner as did subjects in the Pre-Acq and Pre-OV groups during Phase 1. Subjects in the Ctrl-Acq and Ctrl-OV groups that received X–S or AX–S pairings in Phase 1 did not receive any stimulus presentations during this phase.

Phase 3

On Day 10, all subjects received four S–US pairings with an average ITI of 15 min (range = 10–20 min) during a 1-hr session. The onset of the footshock US occurred immediately after the termination of S.

Baseline recovery, Test X, Test A, and Test S

On Days 11 and 12, baseline recovery training was conducted. On each day, subjects received one 60-min session. No nominal stimulus was presented during these sessions. On Day 13, suppression of baseline responding during presentation of CS X was assessed. Each subject received two nonreinforced 30-s presentations of CS X during a 20-min session with the onset occurring 6 and 16 min into the session. On Day 14, suppression to CS A was assessed in the same manner as was done with X on Day 13. On Day 15, suppression of baseline responding during presentation of CS S was assessed. In this session, the duration of presentations of S was expanded from 5 s to 30 s. Although the data of these tests were not the main focus of this experiment, this testing of A and S was conducted to provide data that would complement data from the test of X. The response rate (number of lever presses per minute) during the 60-s period preceding each CS exposure (pre-CS score) and that during the 30-s CS exposure (CS score) was recorded.

Results and Discussion

The suppression ratio (Annau & Kamin, 1961) of each subject was calculated as P/(P + Q) for Days 13–15, where P is the number of lever presses during the two 30-s CS presentations and Q is one half the number of lever presses during the two 60-s pre-CS periods.

Mean suppression ratio to CS X in each group is depicted in Figure 4 (left). Among the three Acq groups, the Post-Acq group showed almost no suppression whereas the Pre-Acq and Ctrl-Acq groups showed robust suppression, indicating that we successfully replicated one of the basic findings in Urushihara, Wheeler, et al. (2004); that is, a recency effect in an outcome-alone exposure paradigm embedded in a sensory-preconditioning preparation was observed. This tendency was reversed in the three OV groups; however, the effect was much smaller. The Post-OV group showed the strongest suppression among the three OV groups. Comparison between the Acq and the OV group in each of the three outcome-exposure conditions clearly shows that an overshadowing effect (i.e., weaker suppression in the OV groups than the corresponding Acq groups) was observed in the Pre and Ctrl conditions, whereas a reversed overshadowing effect (i.e., stronger suppression in the OV group than the corresponding Acq group) was observed in the Post condition. In sum, both the outcome-alone exposure effect and the counteraction between overshadowing and the outcome-alone exposure effect were observed only in the Post condition. These conclusions were supported by the following statistical analyses.

Figure 4.

Mean suppression ratio to the target cue (X; left) and the overshadowing cue (A; right) in Experiment 2. During training, X was paired with a surrogate outcome (S, which was later paired with a foot-shock US), either alone (in the Acq groups) or in compound with an overshadowing cue (A; in the OV groups). S-alone exposures were administered before X → S pairings in the Pre groups and after X → S pairings in the Post groups, whereas no S-alone exposure was administered in the Ctrl groups. Testing was conducted after a brief (4-day) retention interval in Experiment 2. Error bars represent the standard error of the mean.

Prior to any other analyses, it was important to determine whether the control subgroups, which differed in the phase in which X–S or AX–S pairings were conducted, differed in their suppression scores. Thus, a comparison of suppression ratios was first conducted between the two subgroups in the Ctrl-Acq group and between the two subgroups in the Ctrl-OV group with a 2 (subgroup: Phase 1 vs. Phase 2) x 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA. This ANOVA revealed a main effect of group, F(1, 20) = 5.93, p < .03. Neither the main effect of subgroup nor the interaction was significant (Fs < 1). This indicates that the 2-day difference in retention intervals between these subgroups had no appreciable effect on suppression to CS X. Hence, the data from the subgroups within each control group were pooled for purposes of further analysis.

In order to assess any potential differences in fear of the context, pre-CS lever presses were analyzed. Mean pre-CS lever presses on Day 13 for each group across the two test trials were 21.2, 19.4, 20.8, 19.6, 23.7, and 18.5 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Pre, Post, vs. Ctrl) x 2 (training condition: Acq vs. OV) ANOVA conducted on pre-CS lever presses revealed no significant main effect or interaction (Fs < 1). Hence, one cannot easily attribute any differences among groups in suppression to CS X to the differences in contextual fear. A similar 3 × 2 ANOVA conducted on the suppression ratios for CS X revealed a main effect of outcome exposure, F(2, 66) = 6.65, p < .01, and an interaction between outcome exposure and training condition, F(2, 66) = 11.54, p < .001, but no main effect of training condition, F(1, 66) = 1.65, p > .20. Planned comparisons using the error term from the latter ANOVA revealed that the Post-Acq group suppressed less than the Ctrl-Acq group, F(1, 66) = 29.01, p < .001, but the Pre-Acq group did not differ from the Ctrl-Acq group (F < 1). Additional planned comparisons revealed that the Pre-Acq group suppressed more than the Pre-OV group, F(1, 66) = 8.38, p < .01, and the Ctrl-Acq group suppressed more than the Ctrl-OV group, F(1, 66) = 6.28, p < .02, both indicative of the overshadowing effect in the Pre and Ctrl conditions. Finally, the Post-Acq group suppressed less than the Post-OV group, F(1, 66) = 10.07, p < .01, indicative of a reversed overshadowing effect in this condition.

Mean suppression ratios for the overshadowing stimulus (A) in each group are depicted in Figure 4 (right). Among the three OV groups, the Post-OV group showed weaker suppression than the Pre-OV and Ctrl-OV groups, suggesting a strong outcome-postexposure effect but little outcome-preexposure effect. However, despite the fact that the three Ctrl groups received no treatment with CS A, these groups apparently differed in the same direction as the three OV groups, although the difference among the three conditions was greater in the OV condition than in the Acq condition. Because CS A had not been directly paired with the outcome in the Acq condition, the difference in suppression among the three Acq groups was most likely due to stimulus generalization from CS X. However, the difference in suppression to A among the OV groups is not explicable in terms of stimulus generalization because the pattern of suppression to CS A in these groups was in the opposite direction to that for CS X.

Prior to any further analysis, it was necessary to determine whether the control subgroups, which differed in the phase in which X–S or AX–S pairings were conducted, differed in suppression to CS A. Thus, comparisons of suppression ratios were first conducted between the two subgroups in the Ctrl-Acq group and between the subgroups in the Ctrl-OV group with a 2 (subgroup: Phase 1 vs. Phase 2) x 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA. This ANOVA revealed a main effect of group, F(1, 20) = 6.11, p < .03. Neither the main effect of subgroup nor the interaction was significant, Fs(1, 20) < 2.29, ps > .14. This indicates that the 2-day difference in retention intervals between these subgroups had no appreciable effect. Hence, the data from the subgroups within each control group were pooled for purposes of further analysis.

In order to assess any potential differences in expression of fear to the context, pre-CS lever presses were analyzed. Mean pre-CS lever presses on Day 14 for each group across two test trials were 19.4, 19.3, 24.3, 18.8, 22.5, and 20.6 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Pre, Post, vs. Ctrl) x 2 (training condition: Acq vs. OV) ANOVA conducted on pre-CS scores revealed no significant main effect or interaction, Fs < 1. Hence, one cannot attribute any differences among groups in suppression to CS A to the differences in contextual fear. A similar 3 × 2 ANOVA conducted on the suppression ratios for CS A revealed a main effect of outcome exposure, F(2, 66) = 15.32, p < .001, and training condition, F(1, 66) = 13.98, p < .001, and an interaction between outcome exposure and training condition, F(2, 66) = 3.31, p < .05. Planned comparisons using the error term from the latter ANOVA revealed that the Post-OV group suppressed less than the Ctrl-OV group, F(1, 66) = 19.44, p < .001, but the Pre-OV group did not differ from the Ctrl-OV group (F < 1). Among the three Acq groups, the Post-Acq group suppressed less than the Ctrl-Acq group, F(1, 66) = 5.71, p < .02. There was no difference between the Pre-Acq and the Ctrl-Acq groups (F < 1). These results indicate that the outcome-alone exposure had the usual effect on the overshadowing cue (A), which is not subjected to appreciable cue-competition by CS X because it was decidedly more salient than CS X. However, because there were inexplicable differences among the Acq groups, the data from the test of A in this experiment was not conclusive.

Mean suppression ratios to CS S on Day 15 across two test trials were .07, .04, .04, .08, .08, and .06 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. All groups showed similarly strong suppression. Prior to any other analyses, the suppression in control subgroups was analyzed between the two subgroups in the Ctrl-Acq group and between the subgroups in the Ctrl-OV group with a 2 (subgroup: Phase 1 vs. Phase 2) × 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA. This ANOVA revealed no main effect or interaction (Fs < 1). In order to assess any potential differences in expression of fear to the context, pre-CS lever presses were then analyzed. Mean pre-CS lever presses on Day 15 for each group across two test trials were 22.7, 20.0, 21.1, 20.2, 25.5, and 23.0 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Pre, Post, vs. Ctrl) × 2 (training condition: Acq vs. OV) ANOVA conducted on the pre-CS scores revealed no significant main effect or interaction (Fs < 1). A similar 3 × 2 ANOVA conducted on the suppression ratios for CS S revealed no main effect or the interaction (Fs < 1). These results discount the possibility that S-alone exposure resulted in a CS-alone exposure effect on the S–footshock association rather than, or in addition to, an outcome-alone exposure effect upon X. Urushihara, Wheeler, et al. (2004, experiment 1) provide further support for this conclusion.

Suppression to CS X in Experiment 2 suggests that, in a sensory-preconditioning preparation, outcome-alone exposure counteracts overshadowing treatment only when interference by the outcome-alone exposures is effective based upon recency of outcome-alone exposures relative to the cue–outcome pairings. Specifically, comparisons among Acq groups clearly showed that outcome-alone exposure was effective in the Post condition but not in the Pre condition, and a reversed overshadowing effect was observed only in the Post condition.

Experiment 3

In Experiment 2, we found that counteraction between the outcome-alone exposure effect and overshadowing (as well as the outcome-alone exposure effect in elemental conditioning) was influenced by recency of the outcome-alone exposures relative to the cue–outcome pairings. Urushihara, Wheeler, et al. (2004) also reported that the recency effect in an outcome-alone exposure paradigm in a sensory-preconditioning situation vanished and was replaced by a primacy effect when a long retention interval was interposed before testing (i.e., a recency-to-primacy shift occurred with increasing retention interval). Specifically, they found that outcome postexposure caused stronger interference with responding to the elementally trained target cue than did outcome preex-posure when testing was conducted immediately after training, whereas outcome preexposure caused stronger interference than outcome postexposure when testing was conducted after a 3-week retention interval. This finding suggests that we could enhance the outcome-preexposure effect and weaken the outcome-postexposure effect simultaneously by interposing a long retention interval before testing.

The purpose of Experiment 3 was to investigate further the mechanism of the counteraction between the outcome-alone exposure and overshadowing treatment. Toward this end, in Experiment 3 we replicated Experiment 2 with exactly the same parameters but interposed a 3-week retention interval before testing. Based on the results of Urushihara, Wheeler, et al. (2004), a primacy effect rather than a recency effect was anticipated in the Acq condition (i.e., stronger interference in the Pre-Acq group than in the Post-Acq group). Our main interest was how the counteraction effect observed in Experiment 2 would be influenced by interposing such a long retention interval. According to the extended comparator hypothesis, the counteraction between the outcome-alone exposure and overshadowing treatment occurs only when the context as well as the overshadowing stimulus works effectively as comparator stimuli. Thus, in Experiment 3, although the training parameters were largely the same as in Experiment 2, a reversed overshadowing effect was expected to occur in the Pre condition but not in the Post condition because of a primacy effect. We should point out again, however, that the extended comparator hypothesis as well as conventional contemporary associative theories cannot explain why the recency-to-primacy shift occurs when a long retention interval is interposed before testing because these models do not speak to variability in retrieval of learned associations caused by the mere passage of time.

Method

Subjects and Apparatus

Thirty-six male (204–299 g) and 36 female (172–225 g) experimentally naïve Sprague–Dawley-descended rats bred in our colony served as subjects. They were maintained as in Experiment 2 and assigned to one of six groups, Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV. The apparatus and stimuli were identical to those used in Experiment 2.

Procedure

Baseline training and Phases 1, 2, and 3

On Days 1–10, lever press shaping with two CS-alone exposures and Phase 1–3 training were all conducted in the same manner as in Experiment 2 (see Table 2).

Retention interval

On Days 11–31, all animals spent 21 days in their home cages without any experimental manipulation except for 30-s handling three times per week. The water deprivation schedule was maintained throughout this phase.

Baseline recovery and Test X, Test A, and Test S

On Days 32 and 33, baseline recovery training was conducted as in Experiment 2. On Days 34–36, suppression of baseline responding during presentation of CS X, CS A, and CS S was assessed in the same manner as in Experiment 2.

Results and Discussion

The suppression ratio of each subject was calculated as in Experiment 2. Mean suppression ratio to CS X in each group is depicted in Figure 5 (left). Among the three Acq groups, the Pre-Acq group showed weaker suppression than the Post-Acq and Ctrl-Acq groups, suggesting a primacy effect within the outcome-alone exposure effect. Together with the results in Experiment 2, the present observations successfully replicated the findings of Urushihara, Wheeler, et al. (2004); that is, a recency-to-primacy shift in an outcome-alone exposure paradigm with an increasing retention interval was observed. In contrast, the Pre-OV group showed greater suppression than the Post-OV and the Ctrl-OV; however, the differences were smaller than those among the Acq groups. Visual comparison between the Acq and the OV conditions in each of the three outcome-exposure conditions shows that a tendency toward overshadowing (i.e., weaker suppression in the OV groups than the corresponding Acq groups) was observed in the Post and Ctrl conditions, whereas the opposite tendency (i.e., stronger suppression in the OV group than the comparable Acq group) was observed in the Pre condition. In sum, both the outcome-alone exposure effect and the counteraction between overshadowing and the outcome-alone exposure effect were observed not in the Post condition as in Experiment 2 but in the Pre condition. These conclusions were supported by the following statistical analyses.

Figure 5.

Mean suppression ratio to the target cue (X; left) and the overshadowing cue (A; right) in Experiment 3. During training, X was paired with a surrogate outcome (S, which was later paired with a foot-shock US), either alone (in the Acq groups) or in compound with an overshadowing cue (A; in the OV groups). S-alone exposures were administered before X → S pairings in the Pre groups and after X → S pairings in the Post groups, whereas no S-alone exposure was administered in the Ctrl groups. Testing was conducted after a long (25-day) retention interval. Error bars represent the standard error of the mean.

Prior to any other analyses, suppression in the control subgroups was analyzed with a 2 (subgroup: Phase 1 vs. Phase 2) × 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA. This ANOVA revealed a main effect of group, F(1, 20) = 4.60, p < .05. Neither the main effect of subgroup nor the interaction was significant, Fs(1, 20) < 1.74, ps > .20. Therefore, the data from the subgroups within each control group were pooled for purposes of further analysis.

In order to assess any potential differences in fear of the context, pre-CS lever presses were analyzed. Mean pre-CS lever presses on Day 34 for each group across two test trials were 24.0, 21.9, 20.1, 22.2, 22.3, and 18.0 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Ctrl, Pre, vs. Post) × 2 (training condition: Acq vs. OV) ANOVA conducted on pre-CS lever presses revealed no significant main effect or interaction, Fs(1, 20) < 1.29, ps > .28. A similar 3 × 2 ANOVA conducted on the suppression ratios for CS X revealed a main effect of outcome exposure, F(2, 66) = 8.83, p < .001, and an interaction between outcome exposure and training condition, F(2, 66) = 19.26, p < .001, but the main effect of training condition was not significant, F(1, 66) = 2.35, p > .12. Planned comparisons using the error term from the latter ANOVA revealed that the Pre-Acq group suppressed less than the Ctrl-Acq group, F(1, 66) = 48.03, p < .001, indicating a primacy effect in the outcome-alone exposure effect. The difference between the Post-Acq and the Ctrl-Acq groups fell just short of significance, F(1, 66) = 3.97, p = .05, suggesting that there still remains a weak outcome-postexposure effect even after a 3-week retention interval. Additional planned comparisons revealed that the Pre-OV group suppressed more than the Pre-Acq group, F(1, 66) = 35.24, p < .001, suggesting a reversed overshadowing effect in this condition. The difference between the Ctrl-Acq and Ctrl-OV groups fell just short of significance, F(1, 66) = 3.97, p = .05, indicative of a weak overshadowing effect in this experiment. There was no difference between the Post-Acq and the Post-OV groups, F(1, 66) = 1.66, p > .20.

Mean suppression ratio to CS A in each group is depicted in Figure 5 (right). Among the three outcome-alone exposure conditions, the Pre condition showed weaker suppression than the Post and Ctrl conditions in both the Acq and OV conditions, while the difference among the three conditions was greater in the OV condition than in the Acq condition. Because CS A had not been directly paired with the outcome in the Acq condition, the difference in suppression among three Acq groups was most likely due to stimulus generalization from CS X. However, as in Experiment 2 the difference among the OV groups is not explicable in terms of stimulus generalization because the pattern of suppression to CS A in these groups was in the opposite direction to that for CS X.

Prior to any further analysis, comparisons of suppression ratios to CS A between the two subgroups in the Ctrl-Acq group and between the subgroups in the Ctrl-OV group were conducted with a 2 (subgroup: Phase 1 vs. Phase 2) × 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA. This ANOVA revealed a main effect of group, F(1, 20) = 8.62, p < .01. Neither the main effect of subgroup nor the interaction was significant, Fs(1, 20) < 2.36, ps > .14. This indicates that the 2-day difference in retention intervals between these subgroups had no appreciable effect in this experiment. Hence, the data from the subgroups within each control group were pooled for purposes of further analysis.

In order to assess any potential differences in expression of fear to the context, pre-CS lever presses were analyzed. Mean pre-CS lever presses on Day 35 for each group across two test trials were 25.4, 21.5, 19.5, 21.4, 23.5, and 17.2 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Ctrl, Pre, vs. Post) × 2 (training condition: Acq vs. OV) ANOVA conducted on pre-CS scores revealed no significant main effect or interaction, Fs(2,66) < 1.28. Hence, one cannot attribute any differences between groups in suppression to CS A to the differences in contextual fear. A similar 3 × 2 ANOVA conducted on the suppression ratios for CS A revealed a main effect of outcome exposure, F(2, 66) = 14.42, p < .001, and training condition, F(1, 66) = 16.70, p < .001. The interaction was not significant (F < 1).

Mean suppression ratios to CS S on Day 36 across two test trials were .03, .07, .05, .03. .03, and .04 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. All groups showed similar strong suppression. The analysis of the difference between subgroups in the Ctrl condition with a 2 (subgroup: Phase 1 vs. Phase 2) × 2 (group: Ctrl-Acq vs. Ctrl-OV) ANOVA revealed no main effect or interaction (Fs < 1). In order to assess any potential differences in expression of fear to the context, pre-CS lever presses were then analyzed. Mean pre-CS lever presses on Day 36 for each group across two test trials were 24.1, 22.3, 23.7, 21.9, 23.8, and 19.6 for the Pre-Acq, Post-Acq, Ctrl-Acq, Pre-OV, Post-OV, and Ctrl-OV groups, respectively. A 3 (outcome exposure: Ctrl, Pre, vs. Post) × 2 (training condition: Acq vs. OV) ANOVA conducted on pre-CS scores revealed no significant main effect or interaction, (Fs < 1). A similar 3 × 2 ANOVA conducted on the suppression ratios for CS S revealed no main effect or interaction (Fs < 1). Thus, as in Experiment 2, S-alone exposure in the Pre and Post conditions in this experiment did not produce a CS-alone exposure effect with respect to the S–footshock association.

Several details of the results of Experiment 3 call for further consideration. First, although the differences between the Ctrl-Acq and the Ctrl-OV groups were in the same direction in Experiments 2 and 3, the difference between the two conditions in Experiment 3 appears to be smaller than in Experiment 2 and fell just short of statistical significance. This suggests that, although the parameters used in these two experiments were identical, the overshadowing effect observed in Experiment 3 was weaker than that in Experiment 2. This might be because testing was conducted after a long retention interval in Experiment 3; several prior studies reported that interposing a retention interval before testing caused a loss of the overshadowing effect (e.g., Kraemer, Lariviere, & Spear, 1988; Miller, Jagielo, & Spear, 1990). Thus, it is possible that the longer retention interval in Experiment 3 compared to Experiment 2 resulted in the relatively weaker overshadowing effect in Experiment 3. Second, the Post-Acq group showed a tendency toward an outcome-postexposure effect even after a 3-week retention interval, which was evidenced by the marginally significant difference between the Post-Acq and the Ctrl-Acq groups. But there was no apparent counteraction effect between the outcome-alone exposure and overshadowing in the Post condition. One may view the absence of a counteraction effect in the Post group despite the marginal outcome-alone exposure effect in the Post-Acq group as problematic. However, considering that the outcome-alone exposure effect observed in the Pre-Acq group was much stronger than that in the Post-Acq group, the fact that the counteraction effect was observed only in the Pre condition is not surprising. Although a tendency toward overshadowing rather than counteraction was observed in the Post condition, the difference between the Post-Acq and Post-OV was smaller than that between the two otherwise equivalent Ctrl groups, indicative of weak overshadowing in the Post condition.

The results of Experiment 3 conjointly with those of Experiment 2 clearly suggest that the interaction of outcome-alone exposure with a cue-competition treatment (i.e., counteraction with overshadowing), as well as outcome-alone exposure in an elemental conditioning situation, is governed by the recency-to-primacy principle. In these two experiments, counteraction between outcome-alone exposure and overshadowing treatment was observed only in situations in which strong interference by outcome-alone exposure was expected. Specifically, the counteraction effect was observed only in the Post condition when testing was conducted immediately after training (Experiment 2) and only in the Pre condition when testing was conducted after a long retention interval (Experiment 3). These results are congruent with the prediction by the extended comparator hypothesis that the counteraction effect should occur only when the context and the overshadowing stimulus each work effectively as comparator stimuli.

General Discussion

Research in associative learning has frequently examined factors which modulate behavioral control by a cue that is paired with an outcome in an elemental conditioning situation. This line of research culminated in the identification of numerous variables which influence associative behavioral control, such as cue and outcome intensities (for cue intensity, e.g., Kamin, 1965; for outcome intensity, e.g., Annau & Kamin, 1961); probability that the outcome will follow the cue (e.g., Rescorla, 1968); trial spacing (e.g., Gibbon et al., 1977; Jenkins et al., 1981); cue duration (e.g., Coleman et al., 1986; Pavlov, 1927); and effects of cue- or outcome-alone exposures before (for cue-alone preexposure, e.g., Lubow & Moore, 1959; for outcome-alone preexposure, e.g., Randich & LoLordo, 1979), interspersed with (e.g., Rescorla, 1968), or after the cue–outcome pairings (for cue-alone postexposure, Pavlov, 1927; for outcome-alone postexposure, e.g., Rescorla, 1973). Although these findings have been thoroughly examined individually, researchers have made little effort to determine whether these variables have the same effect on responding to cues conditioned in compound with other cues (i.e., cue-competition situations). Seemingly, there has been an unspoken assumption among researchers that the laws of learning generalize from elemental conditioning situations to cue-competition situations. However, recent studies (e.g., Blaisdell et al., 1998; Stout et al., 2003; Urcelay & Miller, 2006; Urushihara, Stout, et al., 2004) have challenged this intuitive assumption by showing that some of the basic laws of learning found in elemental conditioning situations are reversed in cue-competition situations. These findings have a profound implication for associative learning: To the degree that research concerning basic learning is motivated, at least in part, by the belief that there are fundamental principles of learning that apply across varied situations in the real world, we are in urgent need of clarifying how these laws of learning can be applied to situations which include multiple cues because real-world cues in many cases are experienced as elements of complex compound stimuli.

With these backgrounds, the present experiments demonstrated that US (outcome)-alone exposure and overshadowing treatment counteract each other when these two procedures are combined. Specifically, although each procedure caused substantial interference with conditioned responding to the target cue when conducted separately, the disruption in responding caused by a combination of the two procedures was weaker than that caused by either one alone. This can be taken as another example of the reversal of basic laws of learning in cue-competition situations (e.g., Urushihara, Stout, et al., 2004); that is, the US (outcome)-alone exposure, which is known to degrade responding to the target cue in an elemental conditioning situation, can actually enhance responding to the target cue in a cue-competition situation.

Significantly, a similar phenomenon was observed by Maier, Jackson, and Tomie (1987) in an instrumental conditioning situation. They investigated the interaction of two manipulations that individually interfere with learning of escape responses: pretrain-ing presentations of inescapable shocks and presentations of a stimulus contingent on escape responses during learning. When either of these two manipulations was employed alone, they observed interference with learning of escape behavior, whereas when these two procedures were combined, they observed less interference. Although there are many procedural differences between their study and the current experiments, the effects obtained are seemingly similar.

The present findings as well as other examples of the reversal of the basic laws of learning in a cue-competition situation (Blaisdell et al., 1998; Stout et al., 2003; Urcelay & Miller, 2006; Urushihara, Stout, et al., 2004) are challenges to most associative theories which explain cue-competition phenomena as a division of a limited resource (i.e., associative strength or attention). For example, the Rescorla–Wagner model (Rescorla & Wagner, 1972) explains the US-preexposure effect and the overshadowing effect as a division of the total associative strength supportable by the US between the context and the target cue and between the overshadowing cue and the target cue, respectively. Thus, when the two procedures are combined, the two effects are expected to summate. In contrast, the extended comparator hypothesis (Denniston et al., 2001) predicts a counteraction (see Figure 2). As stated in the introduction, in the framework of the extended comparator hypothesis, multiple cues that have a within-compound association with the target cue simultaneously work as first-order comparator stimuli for the target cue and interfere with responding to the target cue at the time of testing. The effects of these multiple first-order comparator stimuli are presumably summative. However, at the same time the response-degrading effect of a first-order comparator stimulus is also degraded by its own comparator stimuli. Applied to the present case, both the context and the overshadowing stimulus work simultaneously as first-order comparator stimuli. Simultaneously, each works as the second-order comparator stimulus for the target cue because there is also a within-compound association between the context and the overshadowing cue. The response-degrading effect of the context (i.e., outcome-preexposure effect) is attenuated by the overshadowing cue (see Figure 2, left), and at the same time that of the overshadowing cue (the overshadowing effect) is also attenuated by the context (see Figure 2, right). Thus, the total response-degrading effects by the context and the overshadowing cue on the target cue can be smaller than that of either effect alone because of the second-order comparator processes which attenuate response-degrading effects by first-order comparator stimuli.

Although the results of Experiment 1 are, as far as we know, exclusively explicable in terms of the extended comparator hypothesis, Experiments 2 and 3 reveal a limitation of the extended comparator hypothesis. As previously noted, the explanation provided by the extended comparator hypothesis is parameter dependent. In order to explain the current results, one has to assume that in the overshadowing condition the sum of the response-degrading effect of the context (which is down-modulated by the second-order comparator process of the overshadowing cue) and that of the overshadowing cue (which is down-modulated by the second-order comparator process of the context) is smaller than the response-degrading effect caused by either the overshadowing cue or the context alone. If this condition is not fulfilled, the extended comparator hypothesis predicts summation rather than counteraction of the two effects. In other words, whether counteraction or summation occurs is dependent on the strength of the second-order comparator processes relative to the first-order comparator processes. However, because the extended comparator hypothesis as proposed by Denniston et al. (2001) is not a quantitative model but a qualitative model, it cannot anticipate the exact conditions in which the second-order comparator processes are strong enough to cause counteraction. In other words, the extended comparator hypothesis needs to be formulated as a quantitative model in order to overcome the uncertainty of its predictions in complex situations like the current experiments.

Sometimes competing retrieval (SOCR), a mathematical implementation of the extended comparator hypothesis recently proposed by Stout and Miller (2006), addressed this difficulty. In the framework of SOCR, simple associative acquisition and extinction processes are described by the following three equations.

$$\mathrm{\Delta }{V}_{\text{X},\text{Y}}=k_1\cdot x\cdot y\cdot (1-V_{\text{X},\text{Y}})$$ (1)

$$\mathrm{\Delta }{V}_{\text{X},\text{Y}}=k_2\cdot x\cdot (0-V_{\text{X},\text{Y}})$$ (2-1)

$$\mathrm{\Delta }{V}_{\text{X},\text{Y}}=k_2\cdot y\cdot (0-V_{\text{X},\text{Y}})$$ (2-2)

In these equations, V_X,Y represents the associative strength between Stimuli X and Y; k₁ and k₂ are the acquisition and extinction parameters, respectively; and x and y are the salience parameters for Stimuli X and Y, respectively. Equation 1 (borrowed from Bush & Mosteller, 1951) defines the increment of associative strength when two stimuli (X and Y) were paired, and Equations 2–1 and 2–2 define the decrements of associative strength when either Stimulus X or Y is presented alone, respectively. Note that unlike other contemporary acquisition-focused models, these associative acquisition processes are not influenced by the presence or absence of other stimuli at the time of training. Based on these associative strengths, responding to a stimulus is determined by the following equation.

$$R_{\text{x}}=x\cdot (V_{\text{X},\text{US}}-k_3\cdot V_{\text{X},\text{A}}\cdot V_{\text{A},\text{US}})$$ (3-1)

Basically, the magnitude of responding to a stimulus (in this case, X) is determined by the product of the salience of that stimulus (x) and the effectiveness of the association between the stimulus and the US (i.e., V_X,US). The effectiveness of association between the target stimulus and the US is modulated by subtraction from the absolute associative strength between the target cue and the US of a comparator term, which is determined by a multiplication of Link 2 (i.e., V_X,A, where A is X’s comparator stimulus); Link 3 (i.e., V_A,US); and k₃, a parameter that reflects the magnitude of the comparator process. In this case, only one comparator stimulus (A) is considered. Thus, Equation 3–1 is a mathematical implementation of the original comparator hypothesis (Miller & Matzel, 1988).

When multiple cues are available to serve as first-order comparator stimuli for the target cue, the effect of each of them is subtracted from the target cue–US association. But the two associations supporting the comparator process in Equation 3–1, that is, Link 2 (V_X,A) and Link 3 (V_A,US), are also subject to their own higher-order comparator processes. Thus, in the case in which there are two comparator stimuli such as in the present case (i.e., context and the overshadowing stimulus, A), the equation looks as follows.

$$\begin{array}{l}{R}_{\text{x}}=\text{x}\cdot (V_{\text{X},\text{US}}-k_3\cdot (V_{\text{X},\text{A}}-k_3\cdot V_{\text{X},\text{Ctx}}\cdot V_{\text{Ctx},\text{ A}})\\ (V_{\text{A},\text{US}}-k_3\cdot V_{\text{A},\text{Ctx}}\cdot V_{\text{Ctx},\text{US}})-k_3\cdot \\ (V_{\text{X},\text{Ctx}}-k_3\cdot V_{\text{X},\text{A}}\cdot V_{\text{A},\text{Ctx}})\cdot \\ (V_{\text{Ctx},\text{US}}-k_3\cdot V_{\text{Ctx},\text{A}}\cdot V_{\text{A},\text{US}}))\end{array}$$ (3-2)

It is important to note that the same parameter determines the effectiveness of the first- and the second-order comparator processes (k₃), but second-order comparator processes will depend on k₃ squared.

A simulation of Experiment 1 by SOCR is depicted in Figure 6. In this simulation, all conditioning treatments in the conditioning phase were divided into 720 30-s segments, and increments or decrements in associative strengths were calculated according to whether stimuli were present or absent in each segment. Salience of each stimulus was set at 0.8, 0.8, 0.6, and 0.3 for the US, CS A, CS X, and the context, respectively.¹ Fixed variables, k₁, k₂, and k₃, were 0.8, 0.001, 0.8, respectively. For simplicity, USs were presented once per 18 segments without ITI variance. Simulation with these parameters results in the following associative strengths at the end of training in each condition: V_X,US = 0.86 in all four conditions; V_A,US = 0.96 and 0.0 in the OV and Acq conditions, respectively; V_Ctx,US = 0.57 and 0.98 in the Ctrl and Pre conditions, respectively; V_X,A = 0.86 and 0.0 in the OV and Acq conditions, respectively; V_X,Ctx 0.46 in all four conditions; and V_A,Ctx = 0.57 and 0.0 in the OV and Acq conditions, respectively. As can be seen in Figure 6, the counteraction effect between US-preexposure and overshadowing treatment is anticipated with these parameters. Note that if we were to use a smaller parameter for the effectiveness of the comparator processes, k₃, the simulation yields a summative rather than a counteraction effect. However, in the present series and other studies, we find that this or a similar value of k₃ is needed to correctly anticipate the behavior that is actually observed (Blaisdell et al., 1998; Stout et al., 2003; Urcelay & Miller, 2006; Urushihara, Stout, et al., 2004). Taking into account that several recent studies have found evidence of so-called higher-order comparator effects (e.g., De Houwer & Beckers, 2002; Denniston, Savastano, Blaisdell, & Miller, 2003; Urushihara, Wheeler, Pineño, & Miller, 2005), setting k₃ relatively high seems plausible.

Figure 6.

Simulation of Experiment 1 using Stout et al.’s (2005) mathematical implementation of the extended comparator hypothesis.

Experiments 2 and 3 further investigated the mechanism underlying the counteraction effect between outcome-alone exposure and overshadowing. According to the extended comparator hypothesis, the counteraction effect between outcome-alone exposure and overshadowing treatment occurs only when both of the two competing stimuli (context and CS A) are strong competitors and are associated with each other. This leads to a testable prediction that, if the context loses its potential to compete with the target cue before the time of testing, the counteraction effect should vanish and its place should be taken by a conventional overshadowing effect. Experiments 2 and 3 tested this prediction making use of the recency effect (Experiment 2) and a recency-to-primacy shift with an increasing retention interval (Experiment 3) to control the effectiveness of the context to compete with other cues (Urushihara, Wheeler, et al., 2004). Experiment 2 showed that when testing was conducted soon after training, the conventional outcome-alone exposure effect occurred only in the outcome-postexposure condition but not in the preexposure condition, suggesting a recency effect for outcome-alone exposure. More important, the counteraction effect was observed only in the outcome-postexposure condition, whereas a conventional overshadowing effect was observed in the preexposure condition, thereby suggesting that the counteraction effect occurs only when the outcome-alone exposure is effective. Experiment 3, which used the same parameters as Experiment 2 with a 3-week retention interval before testing, showed that the conventional outcome-alone exposure effect occurred only in the outcome-preexposure condition but not in the postexposure condition, suggesting a recency-to-primacy shift in the outcome-alone exposure paradigm. Furthermore, the counteraction effect occurred only in the preexposure condition, whereas a tendency toward conventional overshadowing was observed in the postexposure condition. Taken together, the outcome-alone exposure effect in a cue-competition situation, as well as that in the elemental conditioning situation, is governed by the recency-to-primacy principle, yet the direction of the influence on responding is in the opposite direction to that seen with elemental conditioning.

The results of Experiments 2 and 3 are congruent with the prediction by the extended comparator hypothesis that the counteraction effect never appears when the context’s potential to compete with other cues is weak at the time of testing. Corroboratory evidence has been found in prior studies (Stout et al., 2003; Urcelay & Miller, 2006). However, a notable difference between the current experiments and these prior two studies is that in the prior studies the effectiveness of the context was controlled by changing the associative status of a competing stimulus (i.e., posttraining extinction of the context). In the present experiments, it was controlled by changing the retrievability of memories for events experienced in different phases of training (i.e., a recency-to-primacy shift) without any further training of either the competing or target cues. As previously mentioned, the extended comparator hypothesis as well as most associative theories cannot explain why the outcome-alone exposure effect obeys the recency-to-primacy principle. In the framework of the extended comparator hypothesis, the identical number of outcome-alone exposures should result in acquisition of the same associative strength between the context and the outcome, regardless of whether they are conducted before or after target training. Moreover, because the extended comparator hypothesis is silent concerning the effect of the passage of time after learning, it cannot explain why the duration of a retention interval modulates the effect of outcome-alone exposure. Thus, the counteraction as well as the outcome-alone exposure effect is incorrectly predicted to be the same in the pre- and postexposure conditions in Experiments 2 and 3. This clearly qualifies the success of the extended comparator hypothesis with respect to the present experiments.

The difficulty of most associative learning theories in explaining the recency-to-primacy shift in the outcome-alone exposure paradigm with elemental training has been discussed by Urushihara, Wheeler, et al. (2004). Most acquisition-focused learning theories presume that successive episodes during training result in the updating of summary statistics such as associative strength, and the final value of the summary statistic(s) acquired in the training phase is retrieved and determines behavior in the testing phase (e.g., Pearce & Hall, 1980; Rescorla & Wagner, 1972; Wagner, 1981). None of the contemporary models in this family of theories can explain the present counteraction effect, much less how that effect is modulated by the retention interval. Performance-focused theories (e.g., Bouton, 1993, 1997; Denniston et al., 2001; Gallistel & Gibbon, 2000; Miller & Matzel, 1988) assume that a greater amount of information is retained from training and these diverse memories interact at the time of testing to determine behavior. These models are somewhat better at accounting for the present data, although no single one can account for all of the results. As previously explained, the extended comparator hypothesis (Denniston et al., 2001) anticipates the counteraction effect between outcome-alone exposure and overshadowing treatment, but cannot itself explain the recency-to-primacy effect observed in Experiments 2 and 3. In contrast, Bouton’s (1997) retrieval interference model anticipates that recently acquired information should be readily retrieved with a short retention interval (because the test context is similar to the last training context), but, as the retention interval increases, recency wanes and primacy prevails in influencing behavior. However, Bouton’s model is able to account for neither the basic outcome-alone effect nor the basic overshadowing effect, much less the counteraction seen when both treatments are administered. A hybrid of the extended comparator hypothesis (for the counteraction effect) and Bouton’s (1997) retrieval model (for the recency-to-primacy shift) could account for the present observations, alas, at the cost of complexity.

In summary, the current study invites two conclusions. First, overshadowing and the outcome-alone exposure effect counteract each other. This finding can be taken as further evidence of the reversal of select laws of learning in cue-competition situations (e.g., Urushihara, Stout, et al., 2004), and it supports the extended comparator framework proposed by Denniston et al. (2001). Second, the counteraction effect (or alternatively it might be called the higher-order comparator effect) can be influenced by recency and primacy of the outcome-alone exposure relative to target training. This finding, together with the findings by Urushihara, Wheeler, et al. (2004), suggests a limitation of summary-statistics models as well as of the extended comparator hypothesis. Other models that allow coexistence of multiple conflicting memories (e.g., Bouton, 1993, 1997; Gallistel & Gibbon, 2000; Rescorla, 1968) can in principle account for these effects, but some modification of each of these models is necessary. The problem is that most of these models generally have difficulty in explaining even basic cue-competition effects (e.g., Bouton, 1993, 1997), and the few exceptions (e.g., Gallistel & Gibbon, 2000) cannot yet explain the counteraction effect. Thus, it appears necessary to introduce variance in retrievability of associations (or more veridical information) at the time of testing into contemporary learning theories.