Contrasting Predictions of the Extended Comparator Hypothesis and Acquisition-Focused Models of Learning Concerning Retrospective Revaluation

Abstract

Three conditioned suppression experiments with rats investigated contrasting predictions made by the extended comparator hypothesis and acquisition-focused models of learning, specifically, modified SOP and the revised Rescorla-Wagner model, concerning retrospective revaluation. Two target cues (X and Y) were partially reinforced using a stimulus relative validity design (i.e., AX-Outcome/ BX-No outcome/ CY-Outcome/ DY-No outcome), and subsequently one of the companion cues for each target was extinguished in compound (BC-No outcome). In Experiment 1, which used spaced trials for relative validity training, greater suppression was observed to target cue Y for which the excitatory companion cue had been extinguished relative to target cue X for which the nonexcitatory companion cue had been extinguished. Experiment 2 replicated these results in a sensory preconditioning preparation. Experiment 3 massed the trials during relative validity training, and the opposite pattern of data was observed. The results are consistent with the predictions of the extended comparator hypothesis. Furthermore, this set of experiments is unique in being able to differentiate between these models without invoking higher-order comparator processes.

Retrospective revaluation refers to changes in the behavioral control of an absent cue following additional reinforced or nonreinforced training of an associated stimulus. For example, researchers have reported recovery of conditioned responding to an overshadowed stimulus when the overshadowing stimulus was subsequently extinguished (Kaufman & Bolles, 1981; Matzel, Schachtman, & Miller, 1985) or to a blocked stimulus when the blocking stimulus was extinguished (Blaisdell, Gunther, & Miller, 1999). Additionally, Cole, Barnet, and Miller (1995) reported recovery of responding in a relative stimulus validity design to the less valid cue when the more valid cue for reinforcement was extinguished. Retrospective revaluation has also been observed when the associated stimulus was reinforced, such as in backward blocking, which results in decreased conditioned responding to the target cue (Denniston, Miller, & Matute, 1996; Miller & Matute, 1996).

The comparator hypothesis (Miller & Matzel, 1988) and its subsequent versions (extended comparator hypothesis [ECH], Denniston, Savastano, & Miller, 2001; sometimes-competing retrieval [SOCR], Stout & Miller, 2007) have accounted for retrospective revaluation by positing that competition between stimuli occurs at the time of testing. In the framework of the ECH, all stimuli that are present during training acquire associations with the outcome and with each other based on the principle of contiguity alone. At test, a comparator process compares the strength of the unconditioned stimulus (US) representation activated directly by the target conditioned stimulus (CS) to the strength of the US representation indirectly activated conjointly by the association between the target and other cues that were present during target training (comparator stimuli) and the association between these other cues and the US. The output of this comparison determines the strength of conditioned responding to the target. The strength of conditioned responding to the target CS is directly related to the strength of the US representation directly activated by the target and inversely related to the US representation indirectly activated by the comparator stimuli. Applied to situations of retrospective revaluation, ECH assumes that extinguishing a stimulus associated with the target cue (i.e., the comparator stimulus) weakens the strength of the indirectly activated representation of the US, and thus responding to the target stimulus increases.

Initially the comparator hypothesis (Miller & Matzel, 1988; the predecessor of the ECH) was unique in anticipating retrospective revaluation. Learning models that focused on acquisition processes (e.g., Mackintosh, 1975; Pearce & Hall, 1980; Rescorla & Wagner, 1972; Wagner, 1981) assumed that there could be no change in the associative status of a cue on a trial on which the cue was not presented. But, subsequent to the initial statement of the comparator hypothesis, some of the acquisition-focused models of learning were revised to account for retrospective revaluation. Van Hamme and Wasserman (1994) revised the Rescorla-Wagner (1972) model such that the associability parameter for the target CS (α) could assume a negative value (rather than the zero value as was assumed by Rescorla and Wagner) on trials in which the target is absent but an associated cue is presented. For this reason, the associative status of the target stimulus changes in the opposite direction relative to that which it does when it is present. With this modification, the Rescorla-Wagner model can account for changes in responding to a target CS after an associate of the target was trained in its absence.

Dickinson and Burke (1996) modified Wagner's SOP (1981) model to allow excitatory bidirectional associations to form between elements that are activated simultaneously into a high activity memory state (A1) or simultaneously into an intermediate activity memory state (A2), and bidirectional inhibitory associations to form between elements when one element was activated in A1 and the other was activated in A2. Thus, after a target CS is reinforced in compound, if an associate of the target is extinguished, representations of the US and the target stimulus presumably are retrieval primed into the A2 memory state, and consequently the target CS-US association is strengthened. Conversely, further reinforcement of an associate results in the representation of the target cue being retrieval primed into A2 and the representation of the US being activated in A1, which leads to an inhibitory association being formed between the target CS and the US, and responding decreases accordingly.

The present series of experiments was conducted to differentiate between predictions concerning retrospective revaluation made by the ECH and these revised acquisition-focused models (revised Rescorla-Wagner of Van Hamme & Wasserman, 1994 and modified SOP [MSOP] of Dickinson & Burke, 1996) concerning responding to two target stimuli that were separately trained in stimulus relative validity preparations and then one of the associated stimuli for each target stimulus was extinguished in compound. The stimulus relative validity effect refers to weaker responding to a target cue X by a true discrimination group which received AX-US trials interspersed with BX- No US trials, relative to responding to X by a pseudodiscrimination group which received 50% reinforcement of both AX and BX trials. The designs of Experiments 1 and 2 were similar with the exception that Experiment 1 was conducted in first-order conditioning and Experiment 2 was conducted in sensory preconditioning using a surrogate outcome during training in place of a biologically significant reinforcer. Experiment 3 was similar to Experiment 2 except that relative validity training trials were spaced in Experiment 2 and massed in Experiment 3. According to the ECH, massing the trials allowed the context to assume an effective comparator role in Experiment 3 that was otherwise extinguished during the intertrial intervals in Experiments 1 and 2 (Stout, Chang, & Miller, 2003). All three experiments used two group designs (Group Exp and Group Ctrl); each group was subdivided based on which target stimulus (CS X or Y) was tested first. The basic parameters for these experiments were taken from Cole et al. (1995). Those authors included a pseudodiscrimination control group to document that these parameters produce a relative validity effect. Such a control group was omitted from the present research both because it had previously been examined and because the goal of the present research was not to examine the relative validity effect per se.

Experiment 1

In Experiment 1, target CSs X and Y were partially reinforced in compound with CSs A and B and CSs C and D, respectively (AX-US/ BX-No US/ CY-US/ DY-No US), such that they were less valid predictors of reinforcement compared to CSs A and C, which were always reinforced, and CSs B and D, which were never reinforced. In a subsequent phase, the reinforced companion stimulus for Y (C) and the nonreinforced stimulus for X (B) were extinguished in compound (BC-) for the experimental groups (see Table 1). According to MSOP, strong responding should be observed to both target stimuli because representations of X (due to its association with B), Y (due to its association with C), and the US (due to its association with C) should be primed into the A2 state, which should strengthen both the X-US and Y-US associations. Similarly, the revised Rescorla-Wagner model makes the prediction that the target CSs should possess equally strong associative strengths because B and C were extinguished in compound. This should result in the US being expected due to C, and, accordingly, the predictive error for the US, given nonreinforcement on the BC- trials, being the same for both target CSs. According to the revised Rescorla-Wagner model, this should result in acquisition of an association to the US proceeding identically for the two target CSs. In contrast, the ECH predicts strong responding to Y but weak responding to X because at test X's reinforced comparator stimulus (A) is still has a strong association to the US, which can suppression expression of the X-US association. However, Y's reinforced comparator stimulus (C) is weakened and as a result the response potential of Y should be relatively strong. All three models predict weak responding to X and Y in the control groups, which did not receive extinction treatment.

Table 1. Design of Experiment 1.

Subgroup	Phase 1 (Spaced ITI)	Phase 2	ECH predictions	MSOP and R R-W predictions
ExpX	(72 AX-US/ 72 BX-) & (72 CY-US/ 72 DY-)	240 BC-	X→ cr	X→ CR
ExpY	(72 AX-US/ 72 BX-) & (72 CY-US/ 72 DY-)	240 BC-	Y→ CR	Y→ CR
CtrlX	(72 AX-US/ 72 BX-) & (72 CY-US/ 72 DY-)	Context only	X→ cr	X→ cr
CtrlY	(72 AX-US/ 72 BX-) & (72 CY-US/ 72 DY-)	Context only	Y→ cr	Y→ cr

Note: CSs A, B, C, and D were click train, tone, white noise, and flashing light, counterbalanced; CSs X and Y were buzzer and SonAlert, counterbalanced. The US was a footshock. “- ” indicates nonreinforcement. Numbers preceding letters indicate total number of trials in that phase. CR indicates strong conditioned suppression; cr indicates weak conditioned suppression.

Method

Subjects

Subjects were 24 male and 24 female, experimentally naive, Sprague-Dawley descended rats obtained from our own breeding colony. Body-weights ranges were 300-390 g for males and 201-254 g for females. Subjects were randomly assigned to one of two groups, each of which was divided into two subgroups (ns = 12), counterbalanced within groups for sex. The animals were individually housed in standard hanging stainless-steel wire-mesh cages in a vivarium maintained on a 16/8-hr light/dark cycle. Experimental manipulations occurred near the middle portion of the light phase. The animals had free access to Purina Lab Chow, whereas water availability was limited to 20 min per day following a progressive deprivation schedule initiated one week prior to the start of the study. From the time of weaning until the start of the study, all animals were handled for 30 s, three times per week.

Apparatus

Twelve identical chambers, each measuring 30 × 25 × 32 cm (l × w × h), were used. Two of the walls of each chamber were Plexiglas and two were sheet metal. The floor was constructed of 0.5 cm diameter stainless steel rods, spaced 2 cm center-to-center, and connected by NE-2 neon bulbs that allowed a 0.5-s, 0.6-mA constant-current footshock to be delivered by means of a high voltage AC circuit in series with a 1.0-MΩ resistor. Each chamber was housed in an environmental isolation chest, which was dimly illuminated by a houselight (#1820 incandescent bulb) mounted on the ceiling of the experimental chamber. On one metal wall of each chamber, there was an operant lever and adjacent to it a niche (4.5 × 4.0 × 4.5 cm) centered 3.3 cm above the floor. A solenoid could deliver 0.04 cc of water into a cup at the bottom of the niche. A 45-Ω speaker mounted on the interior back side of each environmental chest could deliver a low-frequency tone (500 Hz; 10 dB above background). A second 45-Ω speaker mounted on the ceiling of the experimental chamber was used to deliver a click (6/s; 10 dB above background). A third 45-Ω speaker mounted on a sidewall of the chamber was used to deliver a white noise (10 dB above background). Additionally, a speaker mounted in each environment chest was able to deliver a buzzing sound and a SonAlert could deliver a 1900 Hz tone at 4 dB above the 72 dB background sound level provided by ventilation fans in each enclosure. All auditory cues were measured on the C-scale. A flashing light stimulus (0.25 s on/ 0.25 s off) was provided by a 75-W bulb (nominal at 120 VAC but driven at 60 VAC). The bulb was mounted on the interior backside of each environmental chest. The white noise, click train, low tone, and flashing light served as stimuli A, B, C, and D, counterbalanced within groups. The buzzer and SonAlert served as target stimuli X and Y, counterbalanced within groups. All stimuli were 10 s in duration except for the 0.5-s US.

Procedure

Acclimation and shaping

On Days 1-5, all subjects were acclimated to the context and shaped to lever-press for water during daily 60-min sessions. To facilitate magazine training, water delivery was accompanied by a 0.5-s offset of the houselight on these five days. On Days 1 and 2, a fixed-time 2-min schedule of water reinforcement was in force concurrently with a continuous reinforcement schedule. On Day 3, the noncontingent water delivery was discontinued, and subjects were trained on the continuous reinforcement schedule alone. All subjects that recorded fewer than 50 responses on this day were later placed back in the chambers and hand-shaped through successive approximation for 30 min. Three subjects were given this additional training on Day 3. On Days 4 and 5, water was provided on a variable interval 20-sec (VI-20) schedule. The VI-20 schedule of reinforcement was maintained throughout the remainder of the experiment, including testing.

Preexposure

On Day 6, all subjects received two 30-min sessions in which all CSs were presented elementally without reinforcement two times for 10 s each. CSs A, B, C, and D were presented once during each session, and CSs X and Y were presented twice during one or the other session. This was done to discourage later configuring of the stimulus compounds; for logistical reasons, CSs X and Y had to be presented in separate sessions. The order of stimulus presentations for all subjects was click, SonAlert, tone, noise, SonAlert, and light in Session 1 and noise, buzzer, light, buzzer, tone, and click in Session 2. The mean intertrial interval (ITI) for both sessions was 5 min (range: 4-6 min). For this and all other mean ITIs reported, mean ITI includes time from beginning of session to first stimulus onset and time after last stimulus onset to the end of the session. Session 1 took place in the morning, and Session 2 took place in the afternoon. The US was not programmed to occur during these sessions.

Phase 1

On Days 7-30, all subjects received 6 daily reinforced AX trials and 6 daily nonreinforced BX trials, or 6 daily reinforced CY trials and 6 daily nonreinforced DY trials during 60-min sessions. Due to limitations of our equipment, the relative validity training of X and Y took place on alternating days such that the buzzer and SonAlert, which were counterbalanced within subgroups as CSs X and Y, were never trained on the same day. The CSs and US coterminated. Two training schedules were used for relative validity training of X and Y, which were trained identically on both schedules. For both schedules, the mean ITI was 5 min (range: 2-8 min) with a mean time between USs of 10 min. This relatively long time between reinforced trials has been found in preliminary research sufficient to largely extinguish context-US associations, thereby minimizing the role of the context in behavioral control by X and Y.

Phase 2

On Days 31-35, subjects in the two experimental subgroups (ExpX and ExpY) received 48 daily nonreinforced BC trials during 120-min sessions. The mean ITI was 2.5 min (range: 1-4 min). Subjects in the two control subgroups (CtrlX and CtrlY) received equivalent context exposure but did not receive stimulus presentations.

Reshaping

On Days 36-41, all subjects received baseline recovery training on a VI-20-s schedule during daily 60-min sessions. For logistical reasons, it was intended that no nominal stimuli were to be presented during these sessions, including the houselight off signal that had previously been used in Acclimation to help shape the rats. However, due to experimenter error, on Days 36 and 37, the houselight off signal was presented every time the rats received water reinforcement for pressing the lever. On Days 38 through 41, subjects were given additional reacclimation training to recover lever-pressing behavior in the absence of the houselight-off signal.

Test 1

On Day 42, suppression of baseline responding was assessed during presentations of X in subgroups ExpX and Ctrl X and during presentations of Y in subgroups ExpY and CtrlY. Each subject received four nonreinforced 30-s presentations of CS X or Y, depending on its subgroup, initiated at 8, 13, 18, and 24 min into the 30-min session. The response rates (per min) during the 120-s periods preceding each CS exposure (pre-CS score) and that during the 30-s CS exposure (CS score) were recorded. Subjects continued to be reinforced on a VI-20 schedule throughout the test session. A suppression ratio (Annau & Kamin, 1961) for each subject was calculated by the formula P / (P + Q) for Day 42, where P is the number of lever presses during all four 30-s CS test presentations and Q is the number of lever presses during the 120-s immediately prior to the first CS presentation on each day. A pre-CS period four times as long as the CS presentation was adopted in order to reduce the variability in baseline response rate. A suppression ratio of .50 indicates no suppression, indicative of no conditioned fear, and that of 0 indicates complete suppression, indicative of strong conditioned fear.

Test 2

On Day 43, subjects were tested on the alternate test stimulus such that subjects in subgroups ExpX and CtrlX were tested on CS Y, and subjects in subgroups ExpY and CtrlY were tested on CS X. Testing proceeded exactly as described for Test 1.

ECH predicts that subjects in Group Experimental (subgroups ExpX and ExpY) would show weaker responding to X than to Y. Contrastingly, MSOP and the revised Rescorla-Wagner model predict that subjects should show equally strong responding to both X and Y. Both ECH and the revised acquisition-focused models predict weak responding to X and Y in Group Control (subgroups CtrlX and CtrlY), indicative of the relative validity effect.

Results and Discussion

We observed stronger conditioned suppression to test stimulus Y than to X in Group Experimental. Group Control showed no difference in responding to X and Y, and furthermore Group Control showed less suppression to Y than Group Experimental (see Figure 1). This suggests that extinguishing the BC compound differentially changed responding to Y but not to X. The following statistics support these conclusions.

Figure 1.

Experiment 1: Mean suppression ratio of lever pressing in the presence of the target cues. Brackets indicate standard error of means. See Table 1 for the experimental design.

One subject (from Subgroup CtrlX) was eliminated from all analyses because it did not respond throughout either test session. We conducted a 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) mixed analysis of variance (ANOVA) that analyzed lever pressing during the 120 s preceding the first CS presentation on each day to ensure that there were no differences in baseline responding between Groups Experimental and Control. There was no main effect of group or day, nor an interaction between group and day, all Fs < 1.60. Thus, there were no appreciable differences between groups during the 120 s prior to the first CS presentation on each day. Next we conducted a 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) × 2 (Order: XY vs. YX) mixed ANOVA which confirmed that test order did not significantly interact with group, F(1, 43) = 0.14, p > .05.

After confirming that the order of testing was not relevant, we pooled the subgroups across test days and conducted a 2 (Group: Exp vs. Ctrl) × 2 (Test stimulus: X vs. Y) mixed analysis of covariance (ANCOVA) on the CS test data, with baseline lever pressing as the covariant, thereby factoring out any potential differences in baseline behavior. Although there were not any significant baseline differences between groups in this experiment, an ANCOVA was used instead of an ANOVA to maintain consistency with subsequent experiments in which baseline differences were observed. However, a 2 × 2 mixed ANOVA on the present data yielded the same critical interaction as the ANCOVA. The ANCOVA showed a main effect of group, F(1, 43) = 21.19, MSE = .03, p < .01, Cohen's f = .66 (effect size, Myers & Wells, 2003), showing that responding in Group Experimental differed from Group Control. There was not a main effect of test stimulus in the ANCOVA, F(1, 43) = .49, p > .05, but there was such an effect in the ANOVA, F(1, 45) = 6.17, MSE = .02, p < .05, Cohen's f = .33, showing that responding to X was different from responding to Y. Most importantly, we found an interaction between group and test stimulus, F(1, 43) = 17.76, MSE = .03, p < .01, Cohen's f = .60, suggesting that our extinction treatment differentially affected responding to Y but not X in Group Experimental and that responding to X and Y did not differ in Group Control. A planned comparison between suppression to X and to Y in Group Experimental proved significant, F(1, 43) = 23.35, p < .01, demonstrating that extinguishing the BC compound increased suppression to Y more than X. A comparison between Groups Experimental and Control of suppression to Y was also significant, F(1, 43) = 41.23, p < .01, showing that the increased suppression to Y was due to the extinction treatment received by Group Experimental. In contrast, Groups Experimental and Control did not appreciably differ in suppression to X, nor did Group Control differ in suppression to X and Y, ps > .05.

In Experiment 1 we contrasted predictions made by ECH, MSOP, and the revised Rescorla-Wagner model concerning responding to two stimuli that were embedded in a stimulus relative validity design. Group Experimental, which received compound BC extinction treatment following the relative validity training, exhibited stronger conditioned suppression to Y than to X. These results support the predictions of ECH but are not consistent with the pattern predicted by the revised acquisition-focused models.

According to MSOP and the revised Rescorla-Wagner model, both target CSs, X and Y, should have elicited strong conditioned suppression in Group Experimental. In the framework of MSOP, this is because during Phase 1, CSs X and Y should have become associated with B and C, respectively, and the US should have become associated with C. When B and C were extinguished in compound during Phase 2, their representational elements should have been activated to the A1 memory state, and the representational elements of X, Y, and the US should have been retrieval primed to the A2 memory state, due to their associations with B and C. MSOP allows excitatory associations to form between any elements that are concurrently in the A2 state; consequently, it predicts that equally strong responding should have been observed to X and Y.

In the framework of the revised Rescorla-Wagner model, equally strong responding was predicted to X and Y in Group Experimental because, following Phase 1, the associative strengths of X and Y would have been identical given that they received the same relative validity training. The associative strengths of the target stimuli would have remained equal following Phase 2 because the total expectation of the US (ΣV) during extinction was the same for both CSs because B and C were extinguished in compound. Had B and C been extinguished separately, the revised Rescorla-Wagner model would have predicted differential responding at test because the total associative strength present during the extinction trials would have been different. However, B and C were extinguished in compound and consequently, the revised Rescorla-Wagner model incorrectly predicts that X and Y should have had equal associative strengths (and hence equal control of behavior).

The ECH accounts for the results because it postulates that the extinction treatment should weaken the associative strength of comparator stimulus C; consequently, responding to Y should have increased in Group Experimental. However, X's excitatory comparator stimulus was presumably CS A, not CS B, because A was present during reinforced training with X. Thus, the extinction treatment was not expected to affect responding to X; consequently, weak responding was anticipated because the associative strength of X's comparator (A) was very strong due to its being consistently reinforced.

Experiment 2

Experiment 2 replicated Experiment 1 but in a sensory preconditioning preparation to ensure that the same effects could be observed outside of first-order conditioning. This was done in anticipation of Experiment 3, which further examined the effects of Experiment 1 in sensory preconditioning. All predictions remained the same from Experiment 1 to Experiment 2. See Table 2 for experimental design.

Table 2. Design of Experiment 2.

Group	Phase 1 (Spaced ITI)	Phase 2	Phase 3	ECH predictions	MSOP and R R-W predictions
ExpX	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	240 BC-	6 O→US	X→ cr	X→ CR
ExpY	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	240 BC-	6 O→US	Y→ CR	Y→ CR
CtrlX	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	Context only	6 O→US	X→ cr	X→ cr
CtrlY	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	Context only	6 O→US	Y→ cr	Y→ cr

Note: CSs A, B, C, and D were click train, tone, white noise, and flashing light, counterbalanced; CSs X and Y were buzzer and SonAlert, counterbalanced. CS O was a siren. The US was a footshock. “-” indicates nonreinforcement. Numbers preceding letters indicate total number of trials in that phase. CR indicates strong conditioned suppression; cr indicates weak conditioned suppression.

Method

Subjects

Subjects were 24 male and 24 female, experimentally naive, Sprague-Dawley descended rats obtained from our own breeding colony. Mean body weights were 382 g (range: 325-420) for males and 253 g (range: 230-274) for females.

Apparatus

The experimental chambers were the same as those used in Experiment 1. An additional audio device was mounted on the inner sidewall of the environmental chest that could deliver a 5-s siren sound (10 dB above background) to serve as the surrogate outcome during sensory preconditioning. The US was now a 0.8-mA footshock rather than the 0.7-mA footshock of Experiment 1. The increased shock level was intended to compensate for the response attenuation expected due to the use of sensory preconditioning. All stimuli were 10 s in duration except the surrogate outcome, which was 5 s in duration, and the US, which was 0.5 s in duration.

Procedure

Except where noted, all of the procedures were the same as in Experiment 1.

Acclimation and shaping

On Days 1-5, acclimation to the experimental context and lever-press shaping took place as described in Experiment 1.

Preexposure

On Day 6, all subjects received two 30-min sessions in which all CSs were presented two times as described in Experiment 1.

Phase 1

On Days 7-30, all subjects received 6 daily reinforced AX trials and 6 daily nonreinforced BX trials, or 6 daily reinforced CY trials and 6 daily nonreinforced DY trials during daily 60-min sessions. On reinforced trials, the siren was used as the surrogate outcome in place of a footshock US. The termination of the CS compound coincided with the onset of the surrogate outcome. Like in Experiment 1, the relative validity training of X and Y took place on alternating days such that the buzzer and SonAlert, which were counterbalanced within groups as CSs X and Y, were never trained on the same day. Two training schedules were used with the same ITI as Experiment 1.

Phase 2

On Days 31-35, subjects in Group Experimental received extinction of the BC compound, and subjects in Group Control received context exposure, just like in Experiment 1.

Phase 3

On Day 36, all subjects received six first-order conditioning trials during a 60-min session in which the surrogate outcome (siren) was paired with the footshock. Mean ITI was 10 min (range: 7-13 min). Footshock onset occurred at termination of the siren.

Reshaping

On Days 37-38, all subjects received baseline recovery training in daily 60-min sessions.

Testing

On Day 39, suppression of baseline responding was assessed during presentations of X in subgroups ExpX and Ctrl X and during presentations of Y in subgroups ExpY and CtrlY. Testing proceeded exactly as described for Experiment 1. On Day 40, subjects were similarly tested on the alternate test stimulus.

Results and Discussion

The same basic effects observed in Experiment 1 were replicated in Experiment 2 (see Figure 2). We observed stronger suppression to Y relative to X in Group Experimental. Group Control showed weak responding to both stimuli. A 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) mixed ANOVA on the pre-CS data assessed baseline differences between days and between groups. There was not a main effect of day nor an interaction between group and day, ps > .05, but there was a main effect of group, F(1, 46) = 6.03, p < .05. Thus, an ANCOVA was used on all of the following analyses to factor out the effects of baseline differences. A 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) × 2 (Order: XY vx. YX) mixed ANCOVA ensured that test order did not interact with group, F(1, 42) = .60, p > .05.

Figure 2.

Experiment 2: Mean suppression ratio of lever pressing in the presence of the target cues. Brackets indicate standard error of means. See Table 2 for the experimental design.

Just like in Experiment 1, we then pooled the test data across both days and performed a 2 (Group: Exp vs. Ctrl) × 2 (Test stimulus: X vs. Y) mixed ANCOVA. This revealed a main effect of group, F(1, 44) = 14.15, MSE = .03, p < .01, Cohen's f = .52, and an interaction between group and test stimulus, F(1, 44) = 18.26, MSE = .02, p < .01, Cohen's f = .60. This showed that the extinction treatment of Phase 2 differentially affected responding to Y between groups. Specifically, Group Experimental showed strong suppression to Y but not to X, and Group Control showed weak responding to both Y and X. A planned comparison between responding to X and Y in Group Experimental confirmed this difference, F(1, 44) = 18.45, p < .01. There was also a difference between Groups Experimental and Control in responding to Y, F(1, 44) = 36.64, p < .01. As expected, there was not a difference between Groups Experimental and Control in responding to X, F(1, 44) = .00, p > .05.

These data replicate the main effects from Experiment 1. This experiment was conducted to ensure that these effects could be observed in sensory preconditioning in anticipation of Experiment 3, which further investigated the central finding of Experiments 1 and 2 in a sensory preconditioning preparation.

Experiment 3

Experiment 3 further examined the divergent predictions made by the ECH and the revised acquisition-focused models concerning conditioned suppression to X and Y following compound extinction of two associated stimuli used in relative validity training. The design was similar to Experiment 2 except that in Phase 1, during which subjects were given relative validity training with X and Y, the trials were massed as opposed to spaced. According to ECH, massing the trials makes the training context an effective comparator stimulus for both X and Y. Thus, both targets had two excitatory comparator stimuli, the punctate companion cues (A and C) and the context. In the framework of ECH, two excitatory comparator stimuli that share a within-compound association can counteract the other's potential to compete with the target CS for behavioral control. Our laboratory has provided several demonstrations of counteraction effects in which two comparator stimuli counteracted one another resulting in strong responding to the target stimulus that would otherwise have been weakened by the presence of only one comparator stimulus (see Wheeler & Miller, in press, for examples and further explanation). For the purpose of the present experiment, it is important to note that we have observed counteractive effects between trial massing and other cue competition phenomena such as overshadowing (Stout et al., 2003) and also overexpectation (Sissons & Miller, 2008). Thus in the present study we expected a counteraction between trial massing and another cue competition phenomena, relative validity, which would add generality to the above-mentioned findings. In the current experiment, the context and the punctate comparator stimuli (A and C) should at least partially counteract each other and, consequently, suppression to X and Y should be moderately strong immediately following relative validity training. However, responding to Y should decrease when one of the comparator stimuli, C, is subsequently extinguished leaving only one effective comparator stimulus (the training context) for Y.

Both MSOP and the revised Rescorla-Wagner model predict that responding should be equally strong to X and Y in Group Experimental for the same reasons these models predicted strong responding in Experiment 1, but they also predict equally weak responding to X and Y in Group Control because they anticipate that overshadowing and the deficit from trial massing should summate, not counteract. Thus the ECH predicts that subjects in Group Experimental (subgroups ExpX and ExpY) would show weaker responding to Y than to X. In contrast, MSOP and the revised Rescorla-Wagner model predict that subjects would show equally strong responding to both X and Y. Additionally, ECH predicts equally strong responding to X and Y in Group Control (subgroups CtrlX and CtrlY), evidence of a counteraction between the punctate stimuli and the context, but MSOP and the revised Rescorla-Wagner model predict equally weak responding to X and Y in Group Control due to summation of two response degrading effects.

Previous experiments have demonstrated that it is very difficult to decrease responding to a stimulus after it has already gained appreciable control over behavior without presenting that cue itself or devaluing the US (Denniston et al., 1996; Miller, Hallam, & Grahame, 1990; Miller & Matute, 1996; Oberling, Bristol, Matute, & Miller, 2000). Given that we expected behavioral control by CS Y to first increase in Phase 1 and then decrease in Phase 2 due to extinction of an associated stimulus, it was necessary to avoid making any of the stimuli biologically relevant until completion of extinction treatment. Thus, the critical treatments of Experiment 3 were embedded within a sensory preconditioning preparation such that an innocuous surrogate outcome was used instead of a biologically significant US during Phase 1 of relative validity training (see Table 3).

Table 3. Design of Experiment 3.

Group	Phase 1 (Massed ITI)	Phase 2	Phase 3	ECH predictions	MSOP and R R-W predictions
ExpX	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	240 BC-	6 O→US	X→ CR	X→ CR
ExpY	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	240 BC-	6 O→US	Y→ cr	Y→ CR
CtrlX	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	Context only	6 O→US	X→ CR	X→ cr
CtrlY	(72 AX→O/ 72 BX-) & (72 CY→O/ 72 DY-)	Context only	6 O→US	Y→ CR	Y→ cr

Note: CSs A, B, C, and D were click train, tone, white noise, and flashing light, counterbalanced; CSs X and Y were buzzer and SonAlert, counterbalanced. CS O was a siren. The US was a footshock. “-” indicates nonreinforcement. Numbers preceding letters indicate total number of trials in that phase. CR indicates strong conditioned suppression; cr indicates weak conditioned suppression.

Method

Subjects

Subjects were 24 male and 24 female, experimentally naive, Sprague-Dawley descended rats obtained from our own breeding colony. Body-weights ranges were 283-417 g for males and 185-250 g for females.

Apparatus

The experimental chambers were the same as those used in Experiment 2. All stimuli were the same as in Experiment 2.

Procedure

Except where noted, all of the procedures were the same as in Experiment 2.

Acclimation and shaping

On Days 1-5, acclimation to the experimental context and lever-press shaping took place as described in Experiment 1.

Preexposure

On Day 6, all subjects received two 30-min sessions in which all CSs were presented two times each as described in Experiment 1.

Phase 1

On Days 7-30, all subjects received 6 daily reinforced AX trials and 6 daily nonreinforced BX trials, or 6 daily reinforced CY trials and 6 daily nonreinforced DY trials during daily 9-min sessions. On reinforced trials, the siren was used as the surrogate outcome in place of a footshock US. The termination of the CS compound coincided with the onset of the surrogate outcome. Like in Experiment 1, the relative validity training of X and Y took place on alternating days such that the buzzer and SonAlert, which were counterbalanced within groups as CSs X and Y, were never trained on the same day. Two training schedules were used for relative validity training of X and Y, which were trained identically on both schedules. For both schedules, the mean ITI was 45 s (range: 15-90 s).

Phase 2

On Days 31-35, subjects in the two experimental subgroups (ExpX and ExpY) received 48 daily nonreinforced BC trials during 15-min sessions for a total of 240 BC- presentations. Mean ITI was 18.75 s. Subjects in the two control subgroups (CtrlX and CtrlY) received equivalent context exposure but no exposure to any nominal stimuli. Given the massed nature and short session length of the experimental treatment, the context was not expected to extinguish in either condition. This was important because the context was presumably one of the two cues driving the counteraction, and extinguishing the context would have the same effect (i.e, undoing the expected counteraction between relative validity treatment and trial massing) as extinguishing an excitatory punctate stimulus.

Phase 3

On Day 36, all subjects received six first-order conditioning trials during a 60-min session in which the surrogate outcome (siren) was paired with a biologically significant footshock. Mean ITI was 10 min (range: 7-13 min). Footshock onset occurred at termination of the siren.

Reshaping

On Days 37-38, all subjects received baseline recovery training in daily 60-min sessions.

Testing

On Day 39, suppression of baseline responding was assessed during presentations of X in subgroups ExpX and Ctrl X and during presentations of Y in subgroups ExpY and CtrlY. Testing proceeded exactly as described for Experiment 1. On Day 40, subjects were similarly tested on the alternate test stimulus.

Results and Discussion

As predicted by the ECH, we observed stronger conditioned suppression to CS X relative to CS Y in Group Experimental. There was equal and relatively strong responding to X and Y in Group Control (see Figure 3). Compared to Experiments 1 and 2, these data suggest that the training context and ecitatory punctate comparator stimuli counteracted each other, but extinguishing Y's excitatory punctate companion stimulus resulted in a decrease in responding to Y. The following analyses support these conclusions.

Figure 3.

Experiment 3: Mean suppression ratio of lever pressing in the presence of the target cues. Brackets indicate standard error of means. See Table 3 for the experimental design.

One subject (from Subgroup CtrlX) was eliminated from all analyses due to lack of responding during both of the baseline periods. A 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) mixed ANOVA that looked at responding during the 120 s preceding the first CS presentation on each day was conducted to ensure that there were no differences in baseline responding between Groups Experimental and Control. There was no effect of group in the pre-CS responding, p > .05. Next a 2 (Group: Exp vs. Ctrl) × 2 (Day: Test 1 vs. Test 2) × 2 (Order: XY vs. YX) mixed ANOVA was conducted, which confirmed that test order did not significantly interact with group, F(1, 43) = 0.80, p > .05.

There was no significant difference in responding between groups during the baseline 120 s immediately preceding the first CS presentation on either test day, F(1, 43) = 0.02, p > .05 for Day 1 and F(1, 43) = 0.17, p > .05 for Day 2. The subgroups were pool across test days, and we conducted a 2 (Group: Exp vs. Ctrl) × 2 (Test stimulus: X vs. Y) mixed ANCOVA on the CS test data. As stated previously, an ANCOVA was used instead of an ANOVA to maintain consistency with Experiment 2; all effects and the interaction that were significant in the ANCOVA were also significant in the ANOVA. The ANCOVA detected a main effect of group, F(1, 43) = 14.18, MSE = .02, p < .01, Cohen's f = .53, showing that responding in Group Experimental differed from Group Control. There was also a main effect of test stimulus, F(1, 43) = 5.88, MSE = .01, p < .05, Cohen's f = .32, showing that responding to X was different from responding to Y. Most importantly, there was an interaction between group and test stimulus, F(1, 43) = 27.35, MSE = .01, p < .01, Cohen's f = .75, suggesting that the extinction treatment affected responding to Y but not X in Group Experimental and that responding to X and Y did not differ in Group Control. A planned comparison between suppression to X and to Y in Group Experimental proved significant, F(1, 43) = 45.40, p < .01, demonstrating that extinguishing the BC compound reduced suppression to Y relative to X. A comparison between Groups Experimental and Control of suppression to Y was also significant, F(1, 43) = 29.88, p < .01, showing that the decreased suppression to Y was due to the extinction treatment received by Group Experimental. In contrast, Groups Experimental and Control did not appreciably differ in suppression to X and nor did Group Control differ in responding to X and Y, ps > .05.

These data support the prediction of the ECH that strong conditioned suppression would be observed in Group Control to X and Y and in Group Experimental to X, but Group Experimental would show weak conditioned suppression to Y. According to the ECH, this occurred because the context was made an effective comparator stimulus due to the massed training trials in Phase 1 for all groups, and companion stimuli A and C were also effective comparator stimuli. The context and punctate comparators presumably counteracted each other, allowing the test stimuli to exhibit strong behavioral control in Subgroups ExpX, CtrlX, and CtrlY. However, extinguishing the BC compound in Phase 2 was assumed to have left the context as the only effective excitatory comparator stimulus for Subgroup ExpY, thus allowing the context to exert its full degradative power on Y. The reduction in responding to Y as a result of extinguishing C is a demonstration of mediated learning (specifically, mediated extinction, for discussion, see Holland & Forbes, 1982). These data do not support the predictions by MSOP or the revised Rescorla-Wagner model, which anticipated that subjects in Group Experimental would show equally strong responding to both X and Y, and subjects in Group Control would show equally weak responding to X and Y.

General Discussion

In Experiment 1, we demonstrated recovery of responding to a target stimulus in a stimulus relative validity design by extinguishing the target's reinforced companion stimulus. More importantly, we demonstrated that extinguishing the reinforced stimulus from one relative validity procedure in compound with the nonreinforced companion stimulus of a second relative validity training procedure did not recover responding to the second target stimulus. In Experiment 2 we replicated this effect in sensory preconditioning. In Experiment 3 in which the relative validity training trials were massed, we observed relative to Experiments 1 and 2, what appeared to be a counteraction effect between the punctate comparator stimuli and the diffuse training context. Obviously our claim to having observed a counteraction between the context and the reinforced companion cue has to be qualified because it depends on a between-experiment comparison. Hence, the evidence for a counteraction effect here is at most only suggestive. More central to the focus of this series, in Experiment 3 we showed that extinguishing the reinforced companion stimulus from one relative training procedure in compound with the nonreinforced companion stimulus from a second relative training procedure differentially decreased stimulus control uniquely to the target stimulus trained with the now extinguished reinforced companion stimulus. These results are consistent with the predictions by ECH.

The present observations are not the first demonstration of recovery of responding from a relative validity deficit. Cole et al. (1995) showed that responding to the target cue recovered when the reinforced companion cue was extinguished following relative validity training. This demonstration of retrospective revaluation was consistent with the performance-focused account of the relative validity effect provided by the comparator hypothesis. The current series of experiments replicated the retrospective revaluation effect and extended the findings to show that recovery of responding was specific to the target that had been trained with the revalued excitatory companion stimulus. In other words, behavioral control following extinction treatment changed only for the target for which the reinforced companion cue was extinguished, not to the target for which the nonreinforced companion cue was extinguished. This result may not seem surprising, particularly if one thinks of the design simply as extinction of an overshadowing cue, but it contrasts with the predictions by acquisition-focused models, which were revised to account for retrospective revaluation. The revised Rescorla-Wagner model predicted strong responding to both of the targets because the total predictive error should have been the same for both CSs throughout the experiments. MSOP predicted that responding to the targets should have been equally strong because of the excitatory association presumably formed in the A2 state between the target stimuli and the US. Notably, if new learning about the target cues had occurred during retrospective revaluation treatment (Phase 2) as is assumed by MSOP and the revised Rescorla-Wagner model, equivalent changes in behavioral control should have occurred for both X and Y. At this time, only the ECH anticipates changes in behavioral control by Y but not X.

The ECH can account for the pattern of data from both experiments because it assumes that the response potential of the target stimulus has an inverse relationship to the associative status of its comparator stimuli. In Experiment 1, extinguishing C decreased the response potential of Y's primary excitatory comparator stimulus, but extinguishing B had little effect on the response potential of X because B had gained little excitatory strength during relative validity training. Thus, responding to Y was predicted to increase, whereas responding to X was expected to remain weak. The ECH accounts for the results of Experiment 2 because it predicts that the context and the excitatory punctate comparator (A and C) stimuli should counteract each other in their potentials to decrease responding to the target stimuli. But, extinguishing one of the comparator stimuli (C) should leave only one comparator stimulus (context) associatively strong enough to compete with the target (Y) for behavioral control. As before, extinguishing B should not affect the response potential of X, which was expected to still have strong behavioral control because it was presumably not subject to competition from its excitatory comparator stimuli owing to a counteraction between C and the context. There are numerous other models of basic learning, but none of them anticipate the retrospective revaluation observed in the present studies.

These experiments add to the growing body of literature emphasizing the importance of competition at the time of testing and not just at the time of training. Furthermore, they clearly differentiate between predictions of the ECH and acquisition-focused models that were revised to account for retrospective revaluation. There are numerous examples in the literature in which the predictions by these revised acquisition-focused models have been unsupported. For example, there have been several demonstrations of counteraction effects in which two response-degrading treatments (e.g., latent inhibition and overshadowing) seem to cancel each other out, such that there is strong conditioned responding to the target CS at test, instead of summating to produce an even larger decrement in responding to the target cue, as is predicted by both revised acquisition-focused models (e.g., Blaisdell, Bristol, Gunther, & Miller, 1998). Another example of a failure of the revised acquisition-focused models is in predicting retrospective revaluation of a target after posttraining reinforced or nonreinforced treatment of a cue that was not directly associated with the target. For example, a loss of superconditioning was demonstrated to occur when the excitor used to train the conditioned inhibitor (which was subsequently reinforced with the target cue) was extinguished (Urushihara, Wheeler, Pineño, & Miller, 2005). Denniston, Savastano, Blasidell, and Miller (2003) demonstrated retrospective blocking of a target cue when the target cue was reinforced in compound with an overshadowed blocking stimulus (which was previously reinforced in the presence of a more salient cue) and then the overshadowing cue was extinguished following blocking treatment. In such a situation, the overshadowing had never been paired with the target; yet, its posttraining extinction decreased responding to the target. In such situations, the revised acquisition-focused models predict no change in responding to the target cue because there was not a direct association between it and the cue that was manipulated. But the ECH does anticipate a loss of responding to the target because the overshadowed cue should be an effective second-order comparator stimulus for the target (because it is a first-order comparator stimulus for the blocking stimulus).

In all of these prior demonstrations of failures of the revised acquisition-focused models, the ECH is able to account for the effects by invoking higher-order comparator processes, a feature the revised acquisition-focused models do not have. Notably, this is the first set of experiments which differentiated between predictions by the ECH and the revised acquisition-focused models without having to invoke higher-order processes. Although there was a higher-order process involved in Experiment 2 (the presumed counteraction between the context and reinforced punctate companion stimuli), the focus of the present series of experiments was on the extinction treatment in Phase 2 in which extinction of B and C in compound caused differential retrospective revaluation of the targets X and Y. This did not necessitate higher-order processes, and yet the revised acquisition-focused models still failed to correctly predict differential responding to the target cues.