CS-duration and partial-reinforcement effects counteract overshadowing in select situations

Abstract

Two experiments used rats in a conditioned lick suppression preparation to investigate how the conditioned stimulus (CS)-duration and partial-reinforcement effects (i.e., weakened responding due to conditioning with a CS of longer duration and presenting nonreinforced CSs intermingled with CS-unconditioned stimulus [US] pairings, respectively) interact with overshadowing. Experiment 1 found that when overshadowing treatment was combined with either extended CS duration or partial reinforcement, the response deficit was weaker than when either of these three treatments was administered alone. In Experiment 2, the generality of the findings in Experiment 1 was investigated by replicating it with various US-US intervals. This time counteraction was observed only when both the absolute duration of total CS exposure and the US-US interval were short. The results support neither the view that the ratio between the total CS exposure and total time in the context determines the CS-duration and the partial-reinforcement effects nor the view that these two effects arise from a loss of effectiveness of the excitatory CS-US association during CS-alone exposures in partial reinforcement or early periods of CS exposure with long CSs.

In this report, we focus on two variables that affect Pavlovian conditioning: probability of unconditioned stimulus (US) occurrence with each conditioned stimulus (CS) presentation and the duration of the CS. When the probability of US occurrence per CS presentation is reduced, behavioral control by the CS becomes weaker (the partial-reinforcement effect, e.g., Bouton & Sunsay, 2003; Gibbon, Farrell, Locurto, Duncan, & Terrace, 1980; Papini & Overmier, 1985; Wagner, Siegel, Thomas, & Ellison, 1964). Similarly, when the duration of a CS during conditioning becomes longer, conditioned responding to that CS becomes weaker (the CS-duration effect, e.g., Coleman, Hemmes, & Brown, 1986; Gibbon, Baldock, Locurto, Gold, & Terrace, 1977; Stein, Sidman, & Brady, 1958).

In a simple Pavlovian conditioning situation in which a CS and a US are paired (see the top line [Control] in Figure 1), changing the CS duration is inevitably accompanied by a change of either the interval between USs or the interval between US termination on one trial and CS onset for the next trial. In other words, there are two ways to extend the CS-duration: to extend the CS duration along with the US-US interval while holding the US-CS interval unchanged (see the second line [CS duration A] in Figure 1) and to extend the CS duration with the US-CS interval shortened while holding the US-US interval unchanged (see the third line [CS duration B] in Figure 1). Likewise, decreasing the probability of US presentation with each CS presentation is inevitably accompanied by a decrease in number of US exposures (see the fourth line [Partial reinforcement A] in Figure 1), changes in the US-US interval and number of CS exposures (see the fifth line [Partial reinforcement B] in Figure 1), or changes in the CS-CS interval and the number of CS-exposures (see the bottom line [Partial reinforcement C] in Figure 1). That is, there are three ways to produce the partial-reinforcement treatment: (1) to eliminate some of the US presentations from some of the conditioning trials with the number of CS presentations unchanged, (2) to add CS-alone exposures with the US-US interval extended and the CS-CS interval unchanged (thereby increasing session duration), and (3) to add CS-alone exposures with the US-US interval unchanged and the CS-CS interval shortened. Among these procedures, in the present research we focused on those which keep the US-context relationship unchanged, that is, procedures in which the US-US intervals were unaltered, thereby allowing us to hold session length consistent. Specifically, we used the CS-duration treatment that is produced by extending the CS duration while keeping the US-US interval constant. As a result, the US-CS interval was shortened (CS duration B in Figure 1). We also focused on the partial-reinforcement treatment in which CS-alone trials are added with the US-US interval being held fixed. As a result, the CS-CS interval was shortened (Partial reinforcement C in Figure 1). These choices were made so that we could control the ratio of the CS duration between CS-US pairings to the US-US interval only by varying the total CS duration. If the US-US interval is fixed, two procedures result in an increment of the total duration of the CS presentation between USs and thus a decrement of the ratio of total CS duration between USs to the US-US interval. The only difference between the two procedures is the way in which the total CS duration is increased, that is, making the CS longer on reinforced trials or distributing nonreinforced CS trials of the same duration during the intertrial interval (ITI).

Figure 1.

Various manipulations to create the CS-duration and the partial-reinforcement procedures in simple conditioning situations. The present research used CS duration B and Partial reinforcement C.

There are several different explanations for the partial reinforcement and the CS-duration effects. In this report, because we were investigating these effects in relation to cue competition, we will focus on models which can also account for cue competition effects. Many associative theories explain Pavlovian conditioned responding as a result of acquisition and expression of associations formed between the CS and the US. Although there are many variations of these kinds of theories, most of them assume that an association is formed as the result of contiguous co-occurrences of the two stimuli in question and loses in its effectiveness (either by a decrement of the CS-US association itself or by a development of antagonistic [inhibitory] association between the CS and the US) as the result of presentations of the antecedent stimulus alone. In the framework of these associative theories, the consequences of the partial-reinforcement and the CS-duration treatments are assumed to arise from one or both of two independent effects, weakening of the association between the CS and the US and the development of a CS-context association. Specifically, additional CS presentations during the ITIs without a US presentation (in the case of the partial-reinforcement procedure) or increments in the duration of CS-alone exposure in each CS-US pairing trial (in the case of the CS-duration procedure) results in either a weakening of the CS-US association (e.g., Rescorla & Wagner, 1972) or the formation of an inhibitory association between the CS and the US (e.g., Wagner, 1981). At the same time, increased frequency of the CS or increased durations of conjoint presentation of the CS and the context should result in the formation of a stronger CS-context association.

There are families of models which hypothesize that responding to a CS is determined exclusively by its association with the US (either the absolute strength of the CS-US association [e.g., Mackintosh, 1975; Rescorla & Wagner, 1972; Van Hamme & Wasserman, 1994] or the sum of excitatory and inhibitory associations between them [e.g., Dickinson & Burke, 1996; Pearce & Hall, 1980; Wagner, 1981]). In other words, these models assume a direct mapping of associative strength onto behavior. In the framework of these models, which were referred to as What-You-See-Is-What-You-Have (WYSIWYH) models by Stout and Miller (2007), the effect of partial reinforcement and long CS durations is explained as a loss or impairment of expression of the target CS-US association. Although some of these models (e.g., Wagner, 1981) do incorporate the strengthening of the CS-context association, they only allow the CS-context association to exert its influence by its modulating the acquisition or loss of the CS-US association (either excitatory or inhibitory). In other words, according to these models differences in the CS-context association at the time of testing should not directly affect responding to the CS.

The comparator hypothesis (Denniston, Savastano, & Miller, 2001; Miller & Matzel, 1988) differs from the WYSIWYH models in that it explains various phenomena of associative learning as a result of interactions at the time of behavioral expression (i.e., testing) among multiple associations formed during training. During training, CS-US associations are acquired simply according to principles of spatiotemporal contiguity and salience of the associates. However, in the framework of this model, responding to a target CS is determined not only by the strength of the target CS-US association but also by the interaction of three different associations. The first association is the traditional one between the target CS and the US (Link 1), the second association is between the target CS and other stimuli that were present during CS training (i.e., comparator stimuli; Link 2), and the third association is between the comparator stimulus and the US (Link 3). In a simple conditioning situation in which a single CS is paired with a US, the conditioning context serves as the comparator stimulus for the CS. At test, conditioned responding to the target CS is assumed to reflect a comparison of the US representation directly activated by the target CS (Link 1) and that indirectly activated by the target CS’s comparator stimulus (Links 2 and 3). Excitatory responding to the target CS is positively correlated to the strength of the direct activation of the US representation (Link 1) and negatively correlated to the strength of the indirect activation of the US representation (the product of Links 2 and 3). Note that the comparator hypothesis only postulates excitatory associations; it does not include any inhibitory associations per se.

According to the comparator hypothesis, both the CS-duration and the partial-reinforcement effects are explained by two mechanisms acting conjointly. The first mechanism is the weakening of the target association (i.e., Link 1) caused by repeated nonreinforced presentations of the CS alone during the ITI (the partial reinforcement effect) or increment of the duration of CS-alone exposure (the CS-duration effect), which is shared by many associative theories. Although the comparator hypothesis does not explicitly define the mechanism responsible for this loss of effectiveness of excitatory associations as a result of nonreinforced exposure, this mechanism is implicit considering that the premise of associative acquisition in this model is a simple contiguity rule (see Stout & Miller, 2007, for a formal treatment of extinction in the framework of the comparator hypothesis). The second mechanism is the increment of CS-context association occasioned by these procedures. The mechanism through which the increment of the CS-context association impairs the target responding is, however, different from that hypothesized by some WYSIWYH models (e.g., Wagner, 1981). In the comparator framework, the CS-context association serves as Link 2 when the target CS is tested. When Link 2 is enhanced, the indirectly-activated representation of the US mediated by the product of Links 2 and 3 is enhanced; thus, responding to the target cue is diminished directly by the enhanced status of Link 2 at test.

There exists yet another family of learning theories that can be applied to the partial-reinforcement and the CS-duration effects. These theories, frequently referred to as timing models, explain the effect of CS-duration and the partial-reinforcement treatment in a way different from those of the associative models mentioned above: They assert that responding to a cue (more accurately, the rate of acquisition of conditioned responding) is determined by the ratio of the total CS duration between adjacent CS-US pairings to the US-US interval (Gallistel & Gibbon, 2000; Gibbon & Balsam, 1981; Jenkins, Barnes, & Barrera, 1981). Among these theories, however, only the rate estimation theory (RET) proposed by Gallistel and Gibbon can deal with cue competition effects such as overshadowing. In the framework of RET, the subject decides whether or not to respond to a CS based on the subject’s estimate of the rate of reinforcement during the CS, which is the reciprocal of total CS duration between successive CS-US presentations, relative to the subject’s estimate of the rate of reinforcement during the background (context) alone (i.e., in the absence of the CS). It is assumed that subjects originally have some expectation of reinforcement during the background before they experience any reinforcement. Thus, the estimate of the rate of reinforcement during the background starts from a nonzero value and decreases as the conditioning training continues unless the US is presented without the CS. When the ratio of these two estimates exceeds a decision threshold, the animal begins to respond to the CS. In a simple delay conditioning situation, the estimated rate of reinforcement during the CS does not change while that during the background decreases from its initial value. Subjects start responding to the CS when the estimate of background reinforcement rate has decreased sufficiently. If the CS duration is lengthened with the US-US interval being held fixed, the rate of acquisition is expected to decrease because the estimate of the reinforcement rate during the CS is lower and that during the background decreases more slowly relative to when the CS is short. Likewise, when the CS is presented between CS-US pairings, the rate of acquisition is also predicted to be slower because the estimated reinforcement rate during the CS is lower, while that during the background decreases more slowly relative to when the nonreinforced CSs are not presented.

One eminent prediction of RET is that even if the absolute duration of the CS and the ITI change, responding to the CS should not change as long as the ratio of the CS duration between CS-US pairings to the US-US interval (excluding CS time) is fixed (this is called timescale invariance). As responding to the CS is presumably determined by the estimate of reinforcement rate during the CS relative to that during the background, changing both simultaneously in the same proportion should not result in any change in responding to that CS (Gibbon et al., 1977). This unique prediction clearly differentiates RET from conventional associative models.

In this report, we analyze the effects of the partial reinforcement and long CS duration procedures in an overshadowing situation, focusing on absolute CS duration and the ratio of total CS duration between CS-US pairings to US-US interval. In an overshadowing paradigm, if two conditioned stimuli (CSs) are trained in compound, conditioned responding to the individual CSs is observed to be weaker than when the CSs are trained elementally (Pavlov, 1927). Several recent studies have shown that some variables which modulate Pavlovian conditioning in elemental conditioning situations have opposite effects when the target CS is trained in compound with another CS. For example, the CS-preexposure effect (Blaisdell, Bristol, Gunther, & Miller, 1998; see also Ishii, 1999; Loy & Hall, 2002), the trial-massing effect (Stout, Chang, & Miller, 2003), the degraded-contingency effect (Urcelay & Miller, 2006), and the US (outcome) pre- and postexposure effects (Urushihara & Miller, 2006) are reversed in overshadowing situations relative to elemental conditioning preparations. Moreover, Urushihara, Stout, and Miller (2004) reported that the CS-duration effect counteracts overshadowing. In their experiment, responding to a target CS was stronger when a CS of short duration was trained elementally than when a CS of long duration was trained elementally or a CS of short duration was trained in compound with a more salient cue, suggesting the conventional CS-duration effect and overshadowing, respectively. But when the two treatments were combined, responding to the target CS was stronger than when either alone was administered. The psychological intuition here is that two nontarget competing stimuli can interfere with each other as well as with the target CS, and with appropriate procedures the interference between the two competing stimuli can be greater than the summated direct interference with responding to the target CS.

As mentioned above, there are many different explanations of the response deficit caused by partial-reinforcement and long CSs. However, our review of these different accounts indicates that these two effects are explained by most theories with the same or similar mechanisms despite the differences between explanations offered by each theory. Thus, given the results of Urushihara et al. (2004), the partial reinforcement treatment may well counteract overshadowing. As has been pointed out, in situations in which US-US interval is held fixed, partial reinforcement and the use of long-duration CS have the same effect, that is, both of these procedures result in increasing the total CS duration between CS-US pairings and consequently decreasing the ratio of total CS duration between reinforcements to the US-US interval. In the present experiments, the effects of both long-CS durations and partial reinforcement in overshadowing as well as in elemental conditioning situations were analyzed, manipulating total CS duration between CS-US pairings: either absolute CS duration or that relative to the US-US interval. Investigation of such counteraction is theoretically important because these findings present a serious challenge to widely accepted WYSIWYH models which explain cue competition as a failure of acquisition or loss of excitatory associative effect caused by the division of a limited resource of the outcome (e.g., Rescorla & Wagner, 1972).

EXPERIMENT 1

The central purpose of Experiment 1 was to test the possibility that the partial-reinforcement effect counteracts overshadowing, as has been demonstrated for select other treatments that impair associative responding (Blaisdell et al., 1998; Stout et al., 2003; Urcelay & Miller, 2006; Urushihara & Miller, 2006; Urushihara et al., 2004). In addition, counteraction between the CS-duration effect and overshadowing was further tested against a control condition different from that used in Urushihara et al. In Urushihara et al., the CS-duration effect and its counteraction when administered in conjunction with overshadowing were demonstrated through a comparison with a control condition in which the US-CS interval, rather than the US-US interval, was matched to the experimental condition. In contrast, in the current experiment the US-US interval was kept the same among conditions, consequently changing the US onset to CS onset interval between the long and the short CS conditions. Replication of the effect compared to a different control condition would broaden the generality of the findings of Urushihara et al. Moreover, the total CS duration between USs was equated between the partial-reinforcement condition and long-CS condition. Consequently, the ratio of the total CS duration between reinforcements to US-US interval was also identical across these two groups, despite the very different way in which the CS was presented. Thus, the same behavioral consequence in the two cases was expected by the timing models (e.g., Gallistel & Gibbon, 2000).

The experimental design is summarized in Table 1. A 2 (number of CSs in training: overshadowing [OV] vs. elemental acquisition control [Acq]) × 3 (CS duration: training with the long CS [Long] vs. partial reinforcement training with the short CS [Partial] vs. training with the short CS [Short]) factorial design was used. In the OV condition, subjects received reinforced trials (and in the OV-Partial group nonreinforced trials as well) of the target CS X in compound with more salient overshadowing CS (A), whereas the subjects in three groups in the Acq condition received comparable training with the target cue (X) alone. Orthogonally, the two groups in the Long condition received training with either the compound or elemental CS of long duration (25 sec), the two groups in the Partial condition received partial reinforcement training (10% reinforcement rate) with the CS of short duration (2.5 sec), and the two groups in the Short condition received training with the CS of short duration (2.5 sec), with the number of reinforced trials and the mean US-US interval identical among these conditions (8 trials and 65 sec, respectively). At issue was whether the partial reinforcement as well as the long-duration CS treatment would elicit stronger responding when it was combined with an overshadowing treatment compared to the short-duration target CS was trained elementally with 100% reinforcement, that is, whether the OV-Partial and the OV-Long groups both would exhibit stronger responding than their corresponding control groups (OV-Short and either Acq-Partial or Acq-Long group, respectively).

Table 1.

Design Summary of Experiment 1

Group	Conditioning	CS Duration (sec)	US-US Interval (sec)	Test
Acq-Long	X→US (8)	25	65	X
Acq-Partial	X→US (8) / X (72)	2.5	65	X
Acq-Short	X→US (8)	2.5	65	X
OV-Long	AX→US (8)	25	65	X
OV-Partial	AX→US (8) / AX (72)	2.5	65	X
OV-Short	AX→US (8)	2.5	65	X

Note—Acq, elemental acquisition control; OV, overshadowing; Long, conditioning with CS of long duration; Partial, conditioning with CS of short duration on 10% partial reinforcement schedule; Short, conditioning with CS of short duration; X, target cue; A, overshadowing cue; US, footshock unconditioned stimulus; “→,” sequential pairing of stimuli. CS X was a click train and CS A was a complex tone for all subjects. Numbers in parentheses refer to the number of each trial type.

Method

Subjects

Seventy-two male (263-352 g) and 72 female (178-250 g), experimentally naive, Sprague-Dawley descended rats bred in our colony served as subjects. The animals were randomly assigned to one of six groups (ns = 24): Acq-Long, Acq-Short, Acq-Partial, OV-Long, OV-Short, or OV-Partial, counterbalanced for sex. The experiment was conducted in two exact replications, each of which included 72 animals (ns = 12). The animals were individually housed in wire-mesh cages in a vivarium maintained on a 16:8-h light:dark cycle. Experimental manipulations were conducted approximately midway through the light portion of the cycle. A progressive water deprivation schedule was imposed over 4 days prior to the beginning of the experiment until water availability was limited to 30 min per day. All subjects were handled for 30 sec three times per week from weaning until the initiation of the study.

Apparatus

We used 12 experimental chambers of two different types, 6 identical copies of Chamber Rectangular (R) and 6 identical copies of Chamber V-shaped (V). Chamber R was rectangular in shape, measuring 23.0 × 8.5 × 12.5 cm (l × w × h), and consisted of clear Plexiglas ceiling and side walls and a stainless steel grid floor of 0.48-cm diameter rods separated 1.5 cm apart center to center. The rods were connected by NE-2 neon bulbs that allowed a 0.5-sec, 1.0-mA constant-current footshock to be delivered by means of a high voltage AC circuit in series with a 1.0-MΩ resistor. Each copy of Chamber R was housed in an isolation chest which was dimly illuminated by a 2-W (nominal at 120 VAC) incandescent bulb driven at 60 VAC. The houselight was mounted on the ceiling of the environmental chest approximately 26 cm from the center of the experimental chamber.

Chamber V was a 22.1 cm long box in the shape of a vertical truncated-V (25.3 cm height, 21.3 cm wide at the top, 5.1 cm wide at the bottom). The floor and two long sides consisted of stainless steel sheets, and the ceiling and two short end walls consisted of clear and black Plexiglas, respectively. The floor was composed of two parallel metal plates, each 2.0 cm wide, with a 1.1-cm gap between them, which permitted the delivery of 0.5-sec, 1.0-mA constant-current footshock. Each V-shaped chamber was housed in its own environmental isolation chest which was dimly illuminated by a 7.5-W (nominal at 120 VAC) incandescent houselight driven at 60 VAC mounted on an inside wall of the environmental chest approximately 30 cm from the center of the experimental chamber.

Both Chambers R and V could be equipped with a water-filled lick tube that extended 1 cm from the rear of a cylindrical niche, 4.5 cm in diameter, left-right centered in one short wall, with its axis perpendicular to the wall, and positioned with its center 4.25 cm above the floor of the chamber. Each niche had a horizontal infrared photobeam traversing it parallel to the wall on which the niche was mounted, 1 cm in front of the lick tube. In order to drink from the tube, subjects had to insert their heads into the niche, thereby breaking the infrared photobeam. Thus, we could record when subjects had their heads in the niche with the water tube. Ordinarily, they did this only when they were drinking. Disruption of ongoing drinking by the presentation of a test stimulus served as our dependent variable.

Ventilation fans in each of the 12 environmental chests provided a constant 76-dB (C-scale) background noise. Two 45-Ω speakers mounted on the interior right and left sides of each chest were used to deliver a complex tone (3000 + 3200 Hz, 12 dB [C-scale] above the background) and a click train (6/sec, 6 dB [C-scale] above the background), respectively. Chamber R served as Context Test and Chamber V served as Context Conditioning for half of the subjects in each group, while these relationships were reversed for the other half of the subjects. The physical context was changed between conditioning and testing in order to test the associative effect of the target cue without any contamination by other stimulus, that is, avoiding the excitatory value of the training context from summating with that of the target cue. This allowed us to assess the response potential of the target cue independent of that of the conditioning context. The click train, serving as CS X, and the complex tone, serving as CS A, were either 2.5 or 25.0 sec in duration.

Procedure

Acclimation in Context Test

On Day 1, all subjects were exposed to Context Test for 60 min. The lick tubes were available during this session and baseline licking was established. During the session, subjects also received four presentations of the click and four presentations of the tone. Two of four presentations of each stimulus were of long duration (25 sec) and two were of short duration (2.5 sec). The mean interval between presentations was 7.5 min. The US was not presented. This pretraining exposure was intended to reduce nonassociative responding to the CSs in testing.

Conditioning

On Days 2 and 3, conditioning sessions were conducted during daily 260-sec sessions in Context Conditioning. The lick tubes were not available during these sessions. Subjects in the Acq-Long and OV-Long groups received four daily pairings of the 25-sec CS and the US. Subjects in the Acq-Short and OV-Short groups received four daily pairings of the 2.5-sec CS and the US. Subjects in the Acq-Partial and OV-Partial groups received 36 daily 2.5-sec CS-alone exposures as well as 4 pairings of the 2.5-sec CS and the US, with mean CS onset-CS onset interval of 6.5 sec (range 5-8 sec). The mean US-US interval for all groups was 65 sec. For the Acq-Short, Acq-Long, and Acq-Partial groups, the CS employed in these sessions was CS X alone, whereas for the OV-Short, OV-Long, and OV-Partial groups, the CS employed was a simultaneous compound of CS X and CS A. On the reinforced trials, the 0.5-sec US coterminated with the CS presentation.

Reacclimation

On Days 4-6, all subjects received baseline recovery sessions in Context Test. During these sessions, subjects were placed in Context Test and allowed access to the lick tube for 60 min without any nominal stimulus presentation.

Testing

On Day 7, subjects were tested on CS X in Context Test. Each subject was allowed access to the lick tube and presentation of CS X started when it completed its first 5 cumulative seconds of drinking and remained on for 15 min. Both the time to complete 5 cumulative seconds of drinking in the absence (pre-CS time) and presence (CS time) of CS X were recorded. All test scores were log transformed (base 10) to improve the normality of the within-group variance, thereby better meeting the requirements for parametric statistical tests. As is the custom of our laboratory, subjects that had pre-CS times greater than 60 sec were scheduled to be eliminated from the statistical analyses because of their unusually high fear of Context Test. In practice, no subject met this elimination criterion.

Results and Discussion

Figure 2 depicts the results of Experiment 1. As can be seen, the Acq-Short group showed stronger suppression than the Acq-Long and Acq-Partial groups, suggesting manifestation of the CS-duration effect and the partial-reinforcement effect, respectively. However, among the groups in the OV condition, this tendency was reversed: the OV-Short group showed weaker suppression than the OV-Long and OV-Partial groups, suggesting a reversed CS-duration effect and a reversed partial-reinforcement effect, respectively. The OV-Short group also showed weaker suppression than the Acq-Short group, indicative of overshadowing, whereas the OV-Long and OV-Partial groups showed stronger suppression than the corresponding Acq groups. These conclusions were supported by the following statistical analyses.

Figure 2.

Results of Experiment 1. Mean log latencies to finish the first five cumulative seconds of drinking in the presence of the target cue (X). Error bars represent the standard error of the mean. See text for details.

In order to assess any potential differences in expression of fear to the context, the pre-CS scores were first analyzed with 2 (training CS: Acq vs. OV) × 3 (CS duration: Short vs. Long vs. Partial) × 2 (replication) ANOVA. The ANOVA revealed that none of the main effects nor the interactions was significant (ps > .05). A similar 2 × 3 × 2 ANOVA conducted on the CS test scores yielded a main effect of replication [F(1,132) = 6.84, p < .01], but no interaction with the factor of replication was significant (Fs < 1). Neither the main effect of training CS nor CS duration was significant [F(1,132) = 1.84 and F(2,132) = 1.04, respectively, ps > .17], but a training CS × CS duration interaction was detected [F(2,132) = 14.69, p < .01]. Planned comparisons revealed that suppression in the Acq-Short was stronger than in the Acq-Long and Acq-Partial groups [Fs(1,132) = 13.46 and 20.63, respectively, ps < .01], indicating that conventional detrimental effects of long CS-duration and partial reinforcement occurred. The suppression in the OV-Short group was weaker than that in the OV-Long and OV-Partial groups [Fs(1,132) = 6.01 and 6.33, respectively, ps < .02], indicating that both the partial-reinforcement and the CS-duration treatments enhanced responding in the OV condition. An additional planned comparison revealed that suppression in the Acq-Short group was stronger than in the OV-Short group [F(1,132) = 12.97, p < .01], indicative of a conventional overshadowing effect. Suppression in the Acq-Long group was weaker than in the OV-Long group [F(1,132) = 6.35, p < .02], and suppression in the Acq-Partial group was weaker than in the OV-Partial group [F(1,132) = 11.94, p < .01], consistent with the overshadowing effect being reversed in these two conditions.

The results of Experiment 1 clearly showed that the partial reinforcement effect as well as the CS-duration effect counteracts overshadowing treatment. Specifically, when either of these two procedures was combined with the overshadowing treatment, responding to the target CS was enhanced relative to when either treatment was conducted alone. This can be taken as further evidence of reversal of select basic learning phenomena in overshadowing situations relative to elemental situations. The partial reinforcement treatment counteracted overshadowing as was found with CS-preexposure (Blaisdell et al., 1998), degraded contingency (Urcelay & Miller, 2006), US-alone exposure, (Urushihara & Miller, 2006), massing of training trials (Stout et al., 2003), and the use of long CSs (Urushihara et al., 2004) treatments. Furthermore, the behavioral consequences in the Partial and Long conditions, in which the ratios of the total CS duration between reinforcements to the US-US interval were identical, were apparently the same in the OV condition as well as in the Acq condition, which is consistent with the prediction of the timing models (e.g., Gallistel & Gibbon, 2000). However, these models have difficulty in explaining the counteraction effect itself. For example, in the framework of RET, cue competition effects are explained by a rule called rate additivity. That is, when two or more stimuli are presented in compound, the rate of reinforcement is credited to each element of compound such that the sum of the rate credited to each element equals to the rate credited to the compound, which is determined by the observed rate of reinforcement. According to this view, the rate credited to the target cue in the overshadowing groups should never exceed the rate credited to the target cue in the elemental control groups. Thus, the fact that responding to the target cue in the OV-Long or OV-Partial group was greater than that in the Acq-Long or the Acq-Partial group cannot be explained by RET.

The results of Experiment 1 as well as those of Urushihara et al. (2004) are problematic for WYSIWYH theories (e.g., Rescorla & Wagner, 1972; Wagner, 1981), which explain the CS duration and partial reinforcement effects as a reduction in the strength of the CS-US association because these theories also explain overshadowing as an attenuation of the CS-US association, that is, a failure to acquire an excitatory CS-US association. Thus, when these two manipulations are combined, the two decremental effects should summate, which is contrary to what was observed. As far as we know, the only model which can explain the findings of both Experiment 1 and Urushihara et al., as well as other demonstrations of counteraction between overshadowing and other select treatments known to impair conditioned responding, is the extended comparator hypothesis (Denniston et al., 2001; Figure 3). The extended comparator hypothesis is based on the original comparator hypothesis (Miller & Matzel, 1988, described in the introduction) and further expands on the basic assumption of the original comparator hypothesis by hypothesizing the occurrence of higher-order comparator processes. That is, in the framework of the extended comparator hypothesis, not only is Link 1 modulated by Links 2 and 3, but Links 2 and 3 are assumed to be modulated through a comparator rule similar to that which determines responding to the target cue in the original comparator hypothesis. Thus, the effectiveness of the association between the target CS and the first-order comparator stimulus is determined not only by the strength of the direct association between the target CS and its first-order comparator stimulus, but also by the product of the strength of the association between the target cue and any other potential comparator stimulus (second-order comparator stimuli for Link 2; Link 2.2) and the strength of the association between these other comparator stimuli and the first-order comparator stimulus (Link 2.3). Similarly, the effectiveness of the association between the first-order comparator stimulus and the US is down modulated by the product of the strength of the association between the first-order comparator stimulus and its own comparator stimuli (second-order comparator stimuli for Link 3; Link 3.2) and the strength of the association between the second-order comparator stimuli and the US (Link 3.3). Thus, strong associations between first- and second-order comparator stimuli can down modulate the effectiveness of the first-order comparator stimuli.

Figure 3.

The extended comparator hypothesis (Denniston, Savastano, & Miller, 2001). Ovals represent stimulus representations and rectangles represent the physical stimulus and response. Conditioned responding to the target CS is determined by a comparison between the directly and the indirectly activated US representations at the time of testing: The conditioned responding is positively correlated with the magnitude of the directly activated US representation (Link 1) and negatively correlated with the magnitude of the indirectly activated US representation (the product of Links 2 and 3). In the original comparator hypothesis (Miller & Matzel, 1988, see upper right part of figure excluding the dashed boxes), responding to the target CS is down-modulated only by the absolute strengths of Links 2 and 3, whereas in the extended comparator hypothesis the effectiveness of each of these two comparator links is influenced by its own comparator processes. CS, conditioned stimulus; US, unconditioned stimulus.

Importantly, the extended comparator hypothesis (Denniston et al., 2001) assumes that when multiple possible comparator stimuli exist, each first-order comparator stimulus can simultaneously serve as a second-order comparator stimulus for the other first-order comparator stimuli. For example, when overshadowing treatment is combined with a long CS duration or partial reinforcement, the target CS should have two effective first-order comparator stimuli: the overshadowing stimulus and the training context. At test, these two stimuli both act as the first-order comparator stimuli for the target cue. At the same time, however, these two stimuli serve as second-order comparator stimuli, thereby down modulating the first-order comparator effects of the other comparator stimulus. Consequently, counteraction effect becomes possible. Specifically, the CS-duration effect and the partial-reinforcement effect, both of which presumably arise from strengthening of the target cue-context association (Link 2), should be decreased by the second-order comparator effect of the overshadowing cue because the overshadowing stimulus should form strong associations with the target cue, the US, and the context, all of which down modulate the effectiveness of the product of the target cue-context and the US-context associations. At the same time, the overshadowing effect, which arises from the overshadowing cue’s serving as a first-order comparator stimulus for the target cue, should be decreased by the second-order comparator effect of the context because the context should form a strong association with the target cue and the overshadowing cue, all of which enhance the second-order comparator effect. Importantly, the enhanced responding caused by combining two procedures results from partially releasing the target cue from the first-order comparator processes of both the context and the overshadowing cue, which would otherwise impair the response-eliciting potential of the target cue-US association (Link 1) at the time of testing.

EXPERIMENT 2

As we mentioned in the introduction, there are two mechanisms by which the extended comparator hypothesis explains the CS-duration and partial-reinforcement effects. One of these mechanisms is the establishment of the training context as a comparator stimulus for the target cue (strengthening of Link 2), which we have discussed above. The second mechanism that makes these two treatments decrease responding is a loss of Link 1 (extinction) due to exposures to the early portions of the long CSs on a given trial (with long CSs) or due to CS alone presentations (with partial reinforcement). These two mechanisms allow the comparator hypothesis to explain the decremental effect of these two treatments in elemental conditioning situations. However, these two mechanisms have differential effect on responding to the target cue in overshadowing situations: The establishment of Link 2 for the context allows the extended comparator hypothesis to predict the counteraction observed in Experiment 1, whereas the loss of Link 1 should result in weaker conditioned responding to the target cue also in the overshadowing situations, regardless of the status of comparator stimuli. Thus, if extending the CS duration or partial-reinforcement treatment results in a considerable loss of effectiveness of the target cue-US association (i.e., weakening Link 1) as a result of presenting the CS for a longer duration or presenting it many times during the ITI without the US, the counteraction effect should not occur. We investigated this prediction in Experiment 2 by drastically extending the duration of the CS and the ITI with the ratio of these two parameters fixed.

The goal of Experiment 2 was to differentiate between two theoretical accounts of part of the findings in Experiment 1. Timing models (Gallistel & Gibbon, 2000; Gibbon & Balsam, 1981), although they do not account for counteraction between overshadowing treatment and either partial reinforcement or long CSs, predict that partial-reinforcement and long CSs should have the same behavioral effect provided the ratio of the duration of the total CS presentation between reinforcements to the US-US interval are held constant (i.e., timescale invariance). In contrast, the comparator hypothesis (Miller & Matzel, 1988; Denniston et al., 2001) predicts that, even if the ratio of the total CS duration between reinforcements to the US-US interval is held fixed, a change in the absolute duration of these two parameters can have differential effects on behavioral control in overshadowing situations. Specifically, if both the US-US interval and the duration of the total CS presentation between reinforcements are lengthened, Link 1 (CS-US association) should be weakened because the duration of the CS exposure without the US is lengthened. Presumably, a relatively short CS is critical for the counteraction effect observed in Experiment 1 to occur (although the 25-sec CS in Experiment 1 was labeled as long, this is still a relatively short CS compared to the long CS of Experiment 2); as mentioned above, counteraction is expected only when the target CS-US association is intact. Given impairment of responding to the target cue caused by the loss of the target CS-US association due to long-CS duration or partial-reinforcement treatment, there is little reason to expect counteraction by higher-order comparator processes. That is, even if we hold the ratio of total CS duration between reinforcements to US-US interval fixed, counteraction (i.e., enhanced excitatory responding to the target CS as the result of overshadowing treatment) should be reduced when the absolute CS duration is extended along with the ITI because the target CS-US association should lose its effectiveness as the absolute duration of the CS is extended. As the extended comparator hypothesis is not a mathematical model, the prediction of the model concerning these quantitative differences in strengths of hypothetical associations is unclear. However, SOCR, a mathematical implementation of the extended comparator hypothesis recently proposed by Stout and Miller (2007), enables us to perform quantitative simulations of these predictions, which we discuss in the General Discussion.

The design of Experiment 2 is depicted in Table 2. The intent was to investigate the generality of the findings in Experiment 1 with different timescales. Specifically, we asked whether the three manipulations that decrease the ratio of the US-US interval to the total CS duration between reinforcements—that is, (1) shortening the US-US interval, (2) lengthening the CS duration, and (3) adding nonreinforced trials during US-US interval—are equally conducive to a counteraction effect. Three of the four conditions used, Long, Partial, and Spaced, were identical to the Long, Partial, and Short conditions of Experiment 1, respectively, except that both the CS duration and the US-US interval were 10 times as long as those in Experiment 1. The Massed condition replicated the Long condition in Experiment 1, and it is identical to the Long condition in this experiment except that the CS duration and US-US interval were both only one tenth as long. Importantly, three of four conditions, Long, Partial, and Massed, shared a common ratio of the total CS duration between USs to the US-US interval. Since the parameters used in the Massed condition are identical to those in the Long group in Experiment 1, we expected counteraction of overshadowing in this condition. Our interest was whether the counteraction would also be observed in the Long and Partial conditions. Although the ratio of the CS duration between reinforcements to the US-US interval was identical, the absolute CS duration was longer in the Long and the Partial conditions than in the Massed condition. Thus, according to the extended comparator hypothesis, Link 1 was expected to be weaker in the Long and the Partial conditions than in the Massed condition. This leads to a prediction that the counteraction is less likely to be observed in these two conditions than in the Massed condition.

Table 2.

Design Summary of Experiment 2

Group	Conditioning	CS Duration (sec)	US-US Interval (sec)	Test
Acq-Long	X→US (8)	250	650	X
Acq-Partial	X→US (8) / X (72)	25	650	X
Acq-Spaced	X→US (8)	25	650	X
Acq-Massed	X→US (8)	25	65	X
OV-Long	AX→US (8)	250	650	X
OV-Partial	AX→US (8) / AX (72)	25	650	X
OV-Spaced	AX→US (8)	25	650	X
OV-Massed	AX→US (8)	25	65	X

Note—Acq, elemental acquisition control; OV, overshadowing; Long, conditioning with CS of long duration; Partial, conditioning with CS of short duration on 10% partial reinforcement schedule; Spaced, conditioning with CS of short duration; Massed, conditioning with CS of short duration with short US-US interval; X, target cue; A, overshadowing cue; US, footshock unconditioned stimulus; “→,” sequential pairing of stimuli. CS X was a click train and CS A was a complex tone for all subjects. Numbers in parentheses refer to the number of each trial type.

Method

Subjects and Apparatus

Forty-eight male (232-349 g) and 48 female (177-249 g), experimentally naive, Sprague-Dawley descended rats bred in our colony served as subjects. The animals were randomly assigned to one of eight groups (ns = 12): Acq-Long, Acq-Partial, Acq-Spaced, Acq-Massed, OV-Long, OV-Partial, OV-Spaced, or OV-Massed, counterbalanced for sex. Housing, deprivation, and handling conditions were the same as in Experiment 1. The apparatus was the same as described in Experiment 1.

Procedure

Acclimation in Context Test

On Day 1, all subjects were exposed to Context Test for 40 min. The lick tubes were available and subjects established baseline licking behavior in this session. During this session, subjects received two presentations of the click (X) and two presentations of the tone (A). Each stimulus was presented for 25 sec in duration. The US was not presented. CS X was presented 7 and 35 min into the session and CS A was presented 16 and 26 min into the session. This was intended to reduce nonassociative responding to the CSs in testing and minimize configuring during compound conditioning.

Conditioning

On Days 2 and 3, conditioning sessions were conducted in Context Conditioning. The lick tubes were not available during these sessions. Subjects in the Acq-Spaced and OV-Spaced groups received 4 daily pairings of the 25-sec CS and the US, with the mean US-US interval of 650 sec in a 2,600-sec (43 min and 20 sec) session. Subjects in the Acq-Massed and OV-Massed groups received 4 daily pairings of the 25-sec CS and the US with the mean US-US interval of 65 sec in a 260-sec (4 min and 20 sec) session. Subjects in the Acq-Partial and OV-Partial groups received 36 daily 25-sec CS-alone exposures as well as 4 daily pairings of the 25-sec CS and the US with a mean US-US interval of 650 sec in a 2,600-sec session. The mean CS onset-CS onset interval in the Partial groups was 65 sec (range of 50-80 sec). Subjects in the Acq-Long and OV-Long received 4 daily pairings of the 250-sec CS and the US with a mean US-US interval of 650 sec in a 2,600-sec session. The Acq groups were trained on X alone, whereas the OV groups were trained with a simultaneous compound of X and A. On conditioning trials, the 0.5-sec US coterminated with the CS presentation.

Reacclimation

On Days 4-6, all subjects received baseline recovery sessions in Context Test. During these sessions, subjects were placed in Context Test and allowed access to the lick tubes for 40 min without any nominal stimulus presentation.

Testing

On Day 7, testing of CS X was conducted in Context Test in the same manner as in Experiment 1. Subjects that had pre-CS times greater than 60 sec were scheduled to be eliminated from the statistical analyses because of their unusually high fear of Context Test. In practice, no subject met this elimination criterion. As in Experiment 1, all scores were log transformed to improve normality.

Results and Discussion

Figure 4 depicts mean latencies to complete the first 5 cumulative seconds of drinking in the presence of CS X on Day 7. The Acq-Spaced group showed stronger suppression than the Acq-Long and Acq-Partial groups, suggesting manifestation of the CS duration effect and the partial reinforcement effect, respectively. Although the Acq-Massed group also showed less absolute suppression than the Acq-Spaced group, which is consistent with the frequently observed trial-massing deficit in stimulus control, the difference was not large enough to conclude that the trial-massing effect was successfully obtained. The Acq-Spaced group showed greater suppression than the OV-Spaced group, suggesting conventional overshadowing. The OV-Massed group showed greater suppression than the Acq-Massed group, suggesting counteraction between overshadowing and trial-massing treatment. However, in the Long and Partial conditions, which shared the same ratio of CS duration between reinforcements to US-US interval as the Massed condition, counteraction was not observed. The following statistical analyses confirmed most of these impressions.

Figure 4.

Results of Experiment 2. Mean log latencies to finish the first five cumulative seconds of drinking in the presence of the target cue (X). Error bars represent the standard error of the mean. See text for details.

In order to assess any potential differences in expression of fear to the context, we analyzed pre-CS scores with a 2 (training CS: Acq vs. OV) × 4 (CS-ITI duration: Long vs. Partial vs. Spaced vs. Massed) ANOVA. The ANOVA revealed that none of main effects nor the interaction was statistically significant (p > .05). A similar 2 × 4 ANOVA conducted on the CS scores yielded a main effect of CS-ITI duration [F(3,88) = 14.48, p < .01], and an interaction [F(3,88) = 8.54, p < .01]. Planned comparisons revealed that suppression in the Acq-Spaced group was stronger than in the Acq-Long and Acq-Partial groups [Fs(1,88) = 7.51 and 12.52, respectively, ps < .01], indicative of the CS-duration effect and the partial-reinforcement effect, respectively. However, there was no statistical difference between the Acq-Spaced and Acq-Massed [F(1,88) = 1.14, p > .28], indicative of a failure to obtain the conventional trial-massing effect. Among the OV groups, suppression in the OV-Massed group was stronger than that in the OV-Spaced, OV-Long, and OV-Partial groups [Fs(1,88) = 36.92, 31.91, and 38.15, respectively, ps < .01], which did not differ from each other (Fs < 1). Additional planned comparisons revealed that the suppression in the Acq-Spaced group was stronger than the OV-Spaced [F(1,88) = 8.17, p < .01], indicative of a conventional overshadowing effect, and the suppression in the Acq-Massed group was weaker than the OV-Massed group [F(1,88) = 18.35, p < .01], indicative of counteraction between overshadowing and the trial-massing effect. There was no significant difference in suppression between the Acq-Long and OV-Long groups or between the Acq-Partial and OV-Partial groups (Fs < 1), consistent with the prediction of the extended comparator hypothesis that counteraction would wane.

One apparent problem with the current experiment is that the conventional trial-massing effect (i.e., difference between the Acq-Spaced and Acq-Massed groups) was not clearly observed. The difference between these two groups was in the appropriate direction but did not reach statistical significance. This might be because the subjects in the Massed condition showed some unconditioned fear resulting from experiencing USs with shorter intervals during conditioning, which could have been too weak to be evident in the pre-CS scores but could have summated with conditioned fear to the target cue during testing. Nevertheless, it is reasonable to conclude that the counteraction between the overshadowing and the trial-massing treatment was observed based on the comparisons between the Acq-Massed and OV-Massed groups and between the OV-Massed and OV-Spaced groups. The results of the current experiment, combined with those in Experiment 1, demonstrate that partial reinforcement and long CS durations do not always counteract overshadowing treatment. Specifically, although the ratio of the total CS duration between USs to the US-US interval was held fixed among the Massed, Long, and Partial conditions, counteraction was observed only in the Massed condition. This is congruent with the predictions of the extended comparator hypothesis. According to this model, extending the absolute CS duration simultaneously activates two mechanisms, that is, weakening of Link 1 and enhancing Link 2. These two mechanisms are predicted to cause the same effect, that is, a decrement in responding to the target cue in the elemental conditioning situation. However, in the compound conditioning situation, the behavioral consequence of these two mechanisms is different: Weakening of Link 1 always causes loss of responding to the target cue, whereas enhancing Link 2 can sometimes cause an increment in responding to the target cue due to second-order comparator processes. Because extending the absolute CS duration is expected to result in attenuation of Link 1, responding to the target cue in the OV-Long and the OV-Partial groups, if anything, is expected to be weaker than that in the OV-Massed group despite the ratio of the total CS duration between USs to the US-US interval being the same.

GENERAL DISCUSSION

The current experiments addressed the possibility of counteraction between overshadowing and the partial-reinforcement effect as well as the CS-duration effect and the trial-massing effect. Experiment 1 clearly demonstrated that both the partial reinforcement effect and the CS-duration effect counteract overshadowing. When either the partial reinforcement or the CS-duration treatment was combined with overshadowing treatment, responding to the target cue was stronger than when either partial reinforcement or training with long CSs was conducted elementally. The results of Experiment 1 are taken as further evidence of the reversal of basic phenomena in associative learning in select compound conditioning situations (e.g., Urcelay & Miller, 2006; Urushihara et al., 2004). However, Experiment 2 suggested that the counteraction effect found in Experiment 1 is parameter dependent. When the timescale (i.e., the absolute duration of the CS between USs and the US-US interval) was expanded by a factor of ten, counteraction was no longer observed. Thus, counteraction between overshadowing and the partial-reinforcement or the CS-duration effect seems to be limited to situations in which both the US-US interval and the CS duration between USs are relatively short.

The results of Experiment 1, as well as other examples of reversals of basic phenomena in associative learning in overshadowing situations (Blaisdell et al., 1998; Stout et al., 2003; Urcelay & Miller, 2006; Urushihara et al., 2004; Urushihara & Miller, 2006), are serious challenges to learning models which explain both overshadowing and these basic phenomena as failures of acquisition of the CS-US association (e.g., Rescorla & Wagner, 1972; Wagner, 1981). Contrary to the present results, these models predict summation rather than counteraction of the effects of these procedures. In contrast, the extended comparator hypothesis (Denniston et al., 2001) anticipates the present counteraction effects as the result of higher-order comparator processes. One critical difference between the extended comparator hypothesis and the other models discussed in this paper is that it assumes that the target CS-US association can still be intact even when conditioned responding to the target CS is not observed.

The results of Experiment 2 are problematic not only for the acquisition-focused models (e.g., Rescorla & Wagner, 1972; Wagner, 1981) but also for timing models of conditioning (e.g., Gallistel & Gibbon, 2000). The study clearly demonstrated that timescale invariance is not valid in overshadowing situations. Despite the fact that the ratio of the total CS duration between USs to the US-US interval was identical among the OV-Long, OV-Massed, and OV-Partial groups in Experiment 2, the OV-Massed group showed stronger suppression than the other two groups, and the counteraction effect was observed only in this condition. In contrast, Experiment 2 supported the parameter-dependent prediction of the extended comparator hypothesis (Denniston et al., 2001) that when the CS duration between USs and the US-US interval are both expanded, the counteraction effect between the overshadowing effect and the partial-reinforcement or the CS-duration effect should be attenuated, despite the ratio of these two times being held fixed. This is because the loss of effectiveness of the target cue-US excitatory association (Link 1) becomes greater as the absolute durations of the CS between USs and the US-US interval become longer because of the greater opportunity for weakening of the CS-US association.

A problem for the extended comparator hypothesis, however, is that its predictions concerning counteraction are sometimes parameter dependent. As pointed out in Urushihara and Miller (2006), counteraction is predicted only when overshadowing and the other deleterious effect are of similar strengths and second-order comparator processes are strong. Moreover, the prediction tested in Experiment 2 was dependent on the quantitative status of the hypothesized associations (especially Link 1), despite the fact the extended comparator hypothesis (Denniston et al., 2001) does not provide quantitative predictions. Recently, however, Stout and Miller (2007) proposed a mathematical implementation of the extended comparator hypothesis, which enabled us to conduct a strict quantitative simulation of the present experiments.

The model proposed by Stout and Miller (2007) allows simulation of a series of phenomena on a trial-by-trial basis. On the one hand, this has the merit of simplicity. On the other hand, it is not ideal for situations in which the temporal durations of trials and ITIs differ across conditions. Thus, in order to simulate the present two experiments which include three different CS durations and two different US-US intervals, we modified their model slightly so that we could deal with multiple durations of CSs and ITIs in a consistent manner. In the simulation presented here, all conditioning treatments were divided into 2.5-sec long segments (the shortest CS duration in present experiments), and all increments and decrements of associative strength were calculated based on the presence and absence of each stimulus in that segment. In conjunction with this modification, changes in two parameters were necessary in order to obtain realistic values for each associations: The salience of context was set at 0.17 rather than the 0.35 assumed by Stout and Miller, and k1 (the extinction rate parameter) was set at 0.014 rather than 0.14. These changes were made because if we used the original parameters in the present real-time simulation, all stimuli, especially the context, lost associative strength too rapidly due to decrements in each successive 2.5-sec time bin. All other parameters were the same as those used in Stout and Miller: k2 (proportional impact of comparator process) was set at 0.9, salience of overshadowing cue at 0.71, and salience of target cue at 0.35.

Figure 5 shows the results of each simulation. The top panel depicts a simulation of Experiment 1 (compare to Figure 2). One can see that the simulation closely match the actual data. The middle panel depicts a simulation of Experiment 2 (compare to Figure 4). The results of the simulation in Massed condition in Experiment 2 are identical to those of Long condition in Experiment 1. Importantly, the magnitude of overshadowing and its counteraction, which is represented by the difference between the white and black bars, is much smaller in the Long and Partial conditions than in the Massed and Spaced conditions. These results correspond to our failure to observe differences between the Acq and the OV conditions in the Long and Partial conditions of Experiment 2. The associative strengths between the target cue and the US (i.e., Link 1) obtained from these simulations are, 0.90, 0.89, and 0.97 for the Long, Partial, and Short conditions, respectively, of Experiment 1, and 0.58, 0.50, 0.90, and 0.90 for the Long, Partial, Spaced, and Massed conditions, respectively, of Experiment 2 (these values are equivalent across the Acq and OV conditions). The simulations clearly suggest that strength of Link 1 in the Long, Partial, and Massed conditions differs as a function of total duration of CS exposure, despite response strength in the Acq groups not differing appreciably. As we argued in the introduction to Experiment 2, the response decrement in the Acq-Long and Acq-Partial groups in Experiment 1 and the Acq-Massed group in Experiment 2 is caused mainly by the strong comparator effect of the context while Link 1 is intact. Therefore, counteraction was expected when the first-order comparator effect was attenuated by second-order comparator processes. In contrast, for the Acq-Long and Acq-Partial groups in Experiment 2 (which received far more CS exposure), the associative strength of Link 1 was presumably considerably weakened by the long-duration CS or the partial-reinforcement treatment. Thus, little counteraction was expected in these conditions. The bottom panel of Figure 5 depicts a simulation of the Massed and Spaced conditions of Experiment 1 in Stout et al. (2003), which reported counteraction of overshadowing and trial-massing effect, with a 10-sec CS and 40-sec (Massed) or 960-sec (Spaced) US-US interval. The simulation is fully congruent with their findings. Notably, all three simulations were conducted with identical sets of parameters applied to the CS-duration and the ITI actually used in these experiments. These results, especially the third set, support the appropriateness of our simulations and the parameters we used.

Figure 5.

Results of simulations by SOCR (Stout & Miller, 2007). Top panel: Simulation of Experiment 1. Middle panel: Simulation of Experiment 2. Bottom panel: Simulation of Experiment 1 in Stout et al. (2003). See text for details.

The simulations we presented above illustrate an interesting emergent prediction of SOCR (Stout & Miller, 2007). When we compare the response strengths of the Acq groups which share the same ratio of the total CS duration between USs to the US-US interval, they are roughly the same. Specifically, the response strength of the Acq-Short group in the top panel and that in the Acq-Spaced group in the middle panel are roughly the same, about 0.8, despite the absolute duration of the CS and the US-US interval being ten times longer in the Acq-Spaced group than in the Acq-Short group. Similarly, the response strengths in the Acq-Long and Acq-Partial groups in the top panel and those in the Acq-Long and Acq-Partial groups are all similar, around 0.4, despite the fact that the distributions and absolute durations of the CS between USs and the US-US interval are different. Thus, at least with the current set of parameters, an effect like timescale invariance was simulated. This is superficially surprising in that the strength of Link 1 is different, especially between Long and Partial groups in the top and middle panels. The timescale invariance-like predictions arise because when the CS duration and the US-US interval are both expanded, the weakening of Link 1 is accompanied by a compensating weakening of comparator effect of the context. However, if we expand the absolute duration of the CS and the US-US interval to the extreme, the SOCR predicts that the effectiveness of Link 1 will decline and consequently the CS will finally lose its response potential, even though the ratio between the total CS duration between USs and the US-US interval was fixed. Thus, SOCR can explain and predict timescale invariance to some extent, but it clearly predicts that the effect will no longer be observed if the timescale is made extremely large.

Exposures to either associate alone (i.e., CS or US) are known to have deleterious effects on conditioned responding, regardless of whether they occur before, during, or after the CS-US pairing phase in a Pavlovian conditioning paradigm (the deleterious effects of postpairing US-alone exposures must be qualified, see Miller & Escobar, 2002). The extended comparator hypothesis explains the deleterious effect of exposure to the CS alone as due to enhancement of the CS-context association (Link 2), while US-alone exposures enhance US-context association (Link 3). In other words, each of these manipulations should establish the training context as an effective comparator stimulus for the target cue. These phenomena can be represented in a 2 (associate manipulated: CS or US) × 3 (timing of manipulation: before, during, and after the conditioning phase) matrix. Previous studies together with the results of the current experiments have demonstrated that the phenomena in five of the six cells of the matrix can be reversed in overshadowing situations. Specifically, Blaisdell at al. (1998) demonstrated the counteraction between the CS-preexposure effect and overshadowing, and Urushihara and Miller (2006) demonstrated the US (outcome) pre- and postexposure effect can counteract overshadowing (however, the counteraction between the US-postexposure effect and overshadowing was demonstrated only in a sensory-preconditioning situation). Additionally, Urcelay and Miller (2006) showed that the degraded-contingency effect, which is produced by US-alone exposures intermingled with the CS-US pairings, can counteract overshadowing, and the present Experiment 1 clearly demonstrated the partial-reinforcement effect, which is caused by the CS-alone exposures intermingled with the CS-US pairings, can counteract overshadowing at least in select situations. These facts suggest that the effects of CS-alone and US-alone exposures should be explained with the same theoretical mechanism, despite the fact that most of learning theories treat CSs and USs as playing fundamentally different roles in conditioning. Whether the phenomenon in the sixth cell, extinction, can be reversed in an overshadowing situation, as implausible as it seems, is of future interest.