Intertrial interval as a contextual stimulus: Further analysis of a novel asymmetry in temporal discrimination learning

Abstract

Four experiments investigated discrimination learning when the duration of the intertrial interval (ITI) signaled whether or not the next conditional stimulus (CS) would be paired with food pellets. Rats received presentations of a 10-s CS separated half the time by long ITIs and half the time by short ITIs. When the long ITI signaled that the CS would be reinforced and the short interval signaled that it would not be (Long+/Short−), rats learned the discrimination readily. But when the short ITI signaled that the CS would be reinforced and the long interval signaled that it would not (Short+/Long−), discrimination learning was much slower. Experiment 1 compared Long+/Short− and Short+/Long− discrimination learning with 16-min/4-min or 4-min/1-min ITI combinations. Experiment 2 found no evidence that Short+/Long− learning is inferior because the temporal cue corresponding to the short interval is ambiguous. Experiment 3 found no evidence that Short+/Long− learning is poor because the end of a long ITI signals a substantial reduction in delay to the next reinforcer. Long+/Short− learning may be faster than Short+/Long− because elapsing time involves exposure to a sequence of hypothetical stimulus elements (e.g., A then B), and feature-positive discriminations (AB+/A−) are learned quicker than feature-negative discriminations (A+/AB−). Consistent with this view, Experiment 4 found a robust feature-positive effect when sequentially-presented CSs played the role of elements A and B.

The passage of time can have many effects on learning and memory. An important one is that time can provide a kind of stimulus. For example, the results of many experiments indicate that organisms can use the passage of time after the onset of a stimulus to accurately anticipate presentation of the next reinforcer (e.g., Gallistel & Gibbon, 2000; Gibbon, 1991). In a separate literature, the passage of time has been viewed as part of the context in which learning and remembering occur. From this perspective, “forgetting” that occurs over a retention interval can result from the growing mismatch between the conditions present during learning and those present at retrieval (e.g., Spear, 1978; Smith & Vela, 2001). Bouton (1993) suggested that such an approach can explain spontaneous recovery, the recurrence of extinguished behavior that occurs if time is allowed to pass following extinction. Extinction seems especially dependent on context for retrieval. Spontaneous recovery can thus be viewed as the “renewal effect” that occurs when extinction and testing occur in different temporal contexts.

Bouton and García-Gutiérrez (2006) studied the effects of temporal context in a series of experiments in which the time between trials (the intertrial interval or ITI) provided a cue that signaled whether or not the next presentation of a conditional stimulus (CS) would be reinforced. For example, in one experiment (Experiment 3), several groups of rats received 10-s tone CSs separated half the time by long ITIs (16-min) and half the time by short ITIs (4-min). For Group 16+/4−, tone presentations that followed the 16-min ITI were paired with food pellets, whereas tones that followed the 4-min ITI were not. The rats readily learned to respond more to the tone when it followed the 16-min interval than when it followed the 4-min interval. Control rats that received a pseudodiscrimination in which the same ITIs were each associated with pellets half the time did not show differential responding, indicating that there was no unconditional tendency merely to respond more after long ITIs. And most important, for Group 4+/16−, tones that followed the 4-min ITI were reinforced whereas those that followed the 16-min ITI were not. Surprisingly, this group showed very little evidence of discrimination learning. Thus, although time in the ITI could clearly serve as a discriminative cue, the learning was asymmetrical: It was easier to learn the 16+/4− discrimination than the 4+/16− discrimination. Results of a separate experiment also indicated an asymmetry in the learning of 4+/1− and 1+/4− discriminations. Here the asymmetry was evident in responding during the ITIs themselves; rats in the 4+/1− condition learned to respond more at the end of the 4-min ITI than after the 1-min ITI, whereas rats in the 1+/4− condition were slower to learn their discrimination.

The asymmetry in temporal discrimination learning reported by Bouton and García-Gutiérrez (2006) may have some generality. Todd, Winterbauer, and Bouton (2010) found a similar pattern when a 4-min vs. a 1-min noise (or a 1-min vs. 0.25-min noise) signaled whether a 10-s tone presented at the end of the noise would or would not be paired with food pellets. And in an independent study, Kyd, Pearce, Haselgrove, Amin, and Aggleton (2007) also found that rats were better at using the duration of an auditory cue as a signal for whether or not a food pellet would be delivered 10 s later if the longer duration (e.g., 12 s), rather than the shorter duration (e.g., 3 s), signaled the pellet. The possibility that Long+/Short− discriminations may be learned more rapidly than Short+/Long− discriminations is thus consistent with several types of data.

Bouton and Garcia-Gutierrez (2006) suggested that the apparent asymmetry in temporal discrimination learning is not anticipated by any of several influential models of interval timing (e.g., Church & Broadbent, 1990; Gibbon, Church, & Meck, 1984; Killeen & Fetterman, 1988; Kirkpatrick, 2002; Machado, 1997; see General Discussion for further discussion). Instead, they noted that it was consistent with the idea that time might be coded as a series of elements in which A is followed by B (which is followed by C, etc.). In this approach, a short interval might be represented by A, and a longer interval might be represented by A then B. In these terms, the Long+/Short− discrimination takes the form of an AB+/A−, or feature-positive discrimination. In contrast, the Short+/Long− discrimination takes the form of an A+/AB−, or feature-negative discrimination. Long+/Short− discriminations might be easier to learn because feature-positive discriminations are known to be easier than feature-negative discriminations—the so-called feature-positive effect (e.g., Jenkins & Sainsbury, 1970; Hearst, 1978, 1984). Although Bouton and García-Gutiérrez suggested that A and B might correspond to different strengths of a fading memory trace of the last preceding CS (cf. Staddon & Higa, 1999), the asymmetry is equally consistent with the idea that time is coded as a series of hypothetical elements. We will thus refer to the idea as the “temporal elements” view. A similar idea, applied to the passage of time within the CS rather than in the ITI, appears in several real-time models of conditioning (e.g., Desmond & Moore, 1988; Vogel, Brandon, & Wagner, 2003). For example, Desmond and Moore (1988) represented time as a sequence of elements that were each activated at successive time steps following the onset (or offset) of a CS. Each element then decayed in a manner that produced overlap with elements that followed it (see Kehoe, Horne, Macrae, & Kehoe, 1993, for one representation). In simplified form, the elements in the first steps of an interval might thus be A, then AB, then BC. The elements that are active when the US occurs acquire associative strength according to a version of the Rescorla-Wagner learning rule (Rescorla & Wagner, 1972), which thus allows the development of timed responding on subsequent trials.

The purpose of the present experiments was first to replicate and extend the asymmetry observed by Bouton and García-Gutiérrez (2006), and then to test some alternative explanations of it. In Experiment 1, we compared 16/4 and 4/1 ITI combinations directly and asked whether the asymmetry was maintained when USs were omitted in extinction. In Experiment 2, we asked whether poor discrimination learning in Short+/Long− conditions results from the fact that the temporal cue corresponding to the short ITI is ambiguous in the sense that it predicts reinforcement when it ends in a CS, but nonreinforcement when it is encountered during the course of a long ITI. In Experiment 3, we asked whether poor learning in Short+/Long− results from the fact that the nonreinforced signal after the long ITI might signal a substantial reduction in delay to the next reinforcer. The results failed to confirm implications of either of these possibilities. In a final experiment (Experiment 4), we found a strong feature-positive effect when explicit CSs were tested in the role of hypothetical temporal elements. The data thus support a temporal element account of this novel asymmetry in temporal discrimination learning.

Experiment 1

The first experiment employed a factorial design. Two groups received a Long+/Short− discrimination and two received Short+/Long−. One group from each condition received training with 16- and 4-min ITIs, while the other group received training with 4- and 1-min ITIs. The experiment thus allowed a replication of the temporal asymmetry as well as a direct comparison of learning with these two sets of intervals, which Bouton and García-Gutiérrez (2006) studied in separate experiments. Note also that in this design, the 4-min ITI was a positive cue for groups that had both shorter (Group 4+/1−) and longer (Group 4+/16−) negative ITIs.

The procedures extended the generality of the earlier work in other ways. First, instead of presenting ITIs that varied around means of 16, 4, or 1 min (as in Bouton & García-Gutiérrez, 2006), the intervals in this experiment had fixed values of either 16, 4, or 1. Second, instead of using a procedure in which the different ITIs were double alternated, the present experiment presented them in a quasi-random sequence. Third, the experiment also included an extinction phase in which reinforcement was withheld after the discriminations had been learned. The test in extinction allowed us to determine whether the stimulus controlling performance was indeed the intertrial interval defined as the time between CSs, rather than the time since the last reinforcer.

Method

Subjects

The subjects were 32 female Wistar rats obtained from Charles River Laboratories, St. Constant, Quebec. They were about 90 days old at the start of the experiment, and were individually housed with ad lib water. The rats were maintained at 80% of their free-feeding weights via food restriction throughout the experiment.

Apparatus

The apparatus consisted of two sets of four conditioning chambers housed in separate rooms of the laboratory. Each chamber was housed in its own sound attenuation chamber. All boxes measured 30.5 cm × 24.1 × 23.5 cm (l × w × h). The side walls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of brushed aluminum. The floor was made of stainless steel grids (0.48 in diameter). A recessed 5.08 cm × 5.08 cm food cup was centered in the front wall approximately 2.5 above the level of the floor. Infrared photocells positioned in the cup just behind the plane of the wall monitored entries into the food cup. A 28-V panel light (2.5 cm in diameter) was attached to the wall 10.8 cm above the floor and 6.4 cm to the left of the food cup. The two sets of four boxes had unique features that allow them to be used as different contexts (although they were not used in that capacity here). In one set of boxes, one side wall had black diagonal stripes, 3.81 cm wide and 3.81 cm apart. The ceiling had similarly spaced stripes oriented in the same direction. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). The other set of boxes had no distinctive visual cues, and the grids of the floor were staggered such that odd- and even-numbered grids were mounted in two separate planes, one 0.5 cm above the other.

The chambers were illuminated by two 7.5-W incandescent bulbs mounted to the ceiling of the sound attenuation chamber, approximately 34.9 cm from the grid floor. The CS was a 10-s presentation of a 3000-Hz tone (80 dBA; background was 65 dBA). The US was provided by two 45 mg food pellets (Traditional formula, Research Diets, New Brunswick, NJ) delivered 0.2 s apart. The apparatus was controlled by computer equipment located in an adjacent room.

Procedure

On the first day, the rats were each assigned to a box and received a session of magazine training, in which a total of 30 food pellets were delivered over the course of one 20-min session.

The rats were then randomly assigned to groups with the restriction that boxes be balanced over groups. They then received a single session of conditioning on each of the next 24 days. Two groups (n = 8) received conditioning with 4-min and 16-min fixed ITIs, each ending in a 10-s presentation of the tone CS. For one group (Group 16+/4−), the tone ended in the US when it followed a 16-min ITI, but not when it followed a 4-min ITI. For the other group (Group 4+/16−), the tone ended in the US when it followed a 4-min ITI, but not when it followed a 16-min ITI. The remaining two groups received ITIs of 1-min and 4-min durations. For Group 4+/1−, the tone was reinforced when it followed a 4-min ITI, but not when it followed a 1-min ITI. And for Group 4+/1−, the tone was reinforced when it followed a 1-min ITI, but not when it followed a 4-min ITI.

All groups received an average of four reinforced (R) and four nonreinforced (N) trials during each session, for a total of 8 trials each day. During every four sessions, the groups received sessions in which the R and N trials occurred in RRNNRRNN, NNRRNNRR, RNNRNRRR, or NRRNRNNN sequences. On any day, all groups received the same RN sequence with the corresponding ITIs. Following acquisition, subjects underwent another 4-session cycle of extinction, at which time no USs were administered on any trial.

Data analysis

The computer counted foodcup entries during each 10-s CS and during the 10-s period that preceded the CS (the “pre-CS period”). The measure of responding to the CS was elevation scores of the form e = c – p, where c represents the number of responses recorded during the CS and p the number of responses in the corresponding pre-CS period. Elevation scores have been used extensively in the foodcup entry preparation because they allow separation of responding in the CS from responding during the baseline. Elevation scores and number of responses in the pre-CS periods were analyzed with parallel analyses of variance (ANOVAs) using a rejection criterion of p < .05.

Results

Acquisition

The acquisition results are summarized in Figure 1. Each panel corresponds to a different group, with the 16/4 and 4/1 discriminations in the right- and left-hand columns, and the Long+/Short− discrimination at the top. As the figure suggests, with either the 16/4 or 4/1 ITI combinations, there was a clear asymmetry in the acquisition of the discrimination: the Long+/Short− discriminations were learned more successfully than Short+/Long−. In the elevation scores, which measured responding in the CS relative to baseline, this asymmetry was present, and essentially equivalent, with either combination of ITI. In the pre-CS scores, which measured responding at the end of each ITI, the asymmetry was also present in the rats given the 4/1 intervals. There was less responding in the pre-CS period in the groups given the 16/4 intervals.

Figure 1.

Responding in the CS (elevation scores) and in the pre-CS period (“pre,” in no. responses) over sessions in the acquisition phase of Experiment 1. Different groups are shown in different panels. 16, 4, and 1 = duration of the intertrial interval (mins); +, − = tone reinforced or nonreinforced at the end of the interval, respectively.

The groups’ elevation scores were subjected to an Interval (16/4 vs 4/1) × Discrimination Type (Short+ or Long+) × ITI (reinforced vs nonreinforced) × Session ANOVA. The analysis revealed significant main effects of both ITI, F (1,28) = 48.70, MSE = 7.49, and Session, F (23,644) = 16.37, MSE = 4.49. The ITI effect also increased over sessions, as indicated by the ITI × Session interaction, F (23,644) = 3.65, MSE = 1.79. Most important, the ITI effect interacted with the Discrimination Type factor, F (1,28) = 34.70, MSE = 7.49; thus, whether the animals responded differentially after the ITIs depended on whether the short or the long ITI was reinforced. In addition, this interaction changed over sessions, as indicated by the ITI × Discrimination Type × Session interaction, F (23,644) = 2.35, MSE = 1.79. Simple effect tests exploring the latter interaction revealed that although the Long+/Short− groups had learned a reliable discrimination over the first 12 sessions, F (1,15) = 22.39, MSE = 4.15, the Short+/Long− groups had not, F (1,15) < 1, MSE = 1.11. On the second 12 sessions, there was again a discrimination in the Long+/Short− conditions, F (1,15) = 42.85, MSE = 15.28; the one in the Short+/Long− approached significance, F (1,15) = 4.44, MSE = 2.11, p = .052. None of the other main effects or interactions, including those involving the Interval factor (16/4 vs 4/1), were significant. The asymmetry in discriminative performance was equally evident regardless of whether the discriminations involved 16-min vs. 4-min ITIs or 4-min vs. 1-min ITIs.

Responding in the pre-CS period was analyzed with an identical ANOVA. Here again, there were main effects of ITI, F (1,28) = 31.21, MSE = 1.15, and Session, F (23,644) = 3.02, MSE = 1.37. As in the elevation scores, the difference in responding in the positive and negative ITIs (the ITI effect) was more pronounced in the Long+/Short− groups than the Short+/Long− groups, as indicated by an ITI by Discrimination Type interaction, F (1,28) = 8.35, MSE = 1.15. However, the ITI effect also interacted with Interval (16/4 vs 4/1), F (1,28) = 26.20, MSE = 1.15. Unlike the elevation scores, the asymmetry was more evident in the 4/1 rather than the 16/4 discriminations. Simple effect tests collapsing over session revealed that discriminative performance was significant in Group 4+/1−, F (1,7) = 51.86, MSE = 1.24, but none of the other groups, largest F (1,7) = 3.42, MSE = 3.09. The Interval main effect, F (1,28) = 16.32, MSE = 1.15, and its interaction with session, F (23,644) = 1.59, MSE = 1.37, were also significant. No other main effects or interactions reached the rejection criterion. When it came to the pre-CS scores, there was a strong asymmetry in the Long+/Short− vs. Short+/Long− discriminations when 4- and 1-min intervals were discriminated, but less so when 16- and 4-min intervals were discriminated, presumably because the 16/4 conditions produced substantially lower levels of pre-CS responding.

The results with the elevation scores underwent further analyses to explore the nature of the asymmetry effect further. One a priori analysis compared performance in the 4+/1− and the 4+/16− groups. Although both received reinforced tones after a 4-min ITI, the ability to discriminate clearly depended on whether the nonreinforced ITI was shorter or longer: There was a main effect of the ITI, F (1,14) = 34.13, MSE = 4.75, but it was stronger in Group 4+/1− than 4+/16−, as indicated by the Group × ITI interaction, F (1,14) = 19.38, MSE = 4.75. Both groups responded more on reinforced than nonreinforced trials: Group 4+/1− F (1,7) = 28.75, MSE = 8.67, and Group 4+/16− F (1,7) = 5.92, MSE = 0.83. Two additional analyses separately compared the groups’ responding on the reinforced and nonreinforced trials. The analysis of reinforced trials revealed a main effect of Discrimination Type, F (1,28) = 5.03, MSE = 90.79; the Long+/Short− groups responded more on the reinforced trials than did the Short+/Long− groups. Aside from a Session effect, F (23,644) = 17.08, MSE = 3.35, no other effects approached significance. The analysis of the nonreinforced trials revealed no overall effect of Discrimination Type, F (1,28) <1, MSE = 53.28. but there was a significant Discrimination Type × Session interaction, F (23,644) = 2.85, MSE = 2.82. The interaction indicates that responding on the nonreinforced trials became lower over sessions in the Long+/Short− than Short+/Long− groups. Other than the Session effect, F (23,644) = 7.87, MSE = 2.82, no other effects approached significance. Overall, the results leave little doubt that both Long+/Short− discriminations were learned more readily than their Short+/Long− counterparts, and that the difference resulted from the Long+/Short− groups (a.) responding more on the positive trials and (b.) responding less on the later negative trials.

Extinction

Figure 2 depicts the results during the four extinction sessions in which the rats no longer received pellets at the end of the CS. As the figure suggests, although responding decreased over sessions, discriminative performance continued to be more accurate in the Long+/Short− groups than in the Short+/Long− groups. An Interval (16/4 vs 4/1) × Discrimination Type (Short+ or Long+) × ITI (reinforced vs nonreinforced) × Session ANOVA revealed significant main effects ITI, F (1,28) = 4.15, MSE = 2.19, and Session, F (3,84) = 22.76, MSE = 2.36. Once again, the ITI × Discrimination Type interaction was highly significant, F (1,28) = 22.36, MSE = 2.19. As in acquisition, the interaction took the form of a bigger ITI effect when the Long interval (rather than the short interval) had predicted reinforcement; the Long+/Short− groups responded more in the previously-reinforced than nonreinforced ITI, F (1,15) = 19.89, MSE = 3.07, whereas the Short+/Long− groups showed the opposite tendency, F (1,15) = 4.66, MSE = 1.06. Not surprisingly, the ITI effect also interacted with session, F (3,84) = 3.05, MSE = 2.36. No other effects or interactions reached significance. An analogous ANOVA on the pre-CS scores revealed no reliable differences except for a main effect of Interval, F (1,28) = 5.66, MSE = 3.37, and an ITI × Discrimination Type × Session interaction, F (3,84) = 5.26, MSE = 0.56. These effects reflect the fact that the rats in the 4+/1− group started with the highest responding and best discriminative performance in pre-CS behavior, and that this declined as extinction continued.

Figure 2.

Responding in the CS (elevation scores) and in the pre-CS period (“pre,” in no. of responses) over sessions in the extinction phase of Experiment 1. Different groups are shown in the different panels. 16, 4, and 1 = duration of the intertrial interval (mins); +, 1 = tone reinforced or nonreinforced at the end of the interval, respectively.

Discussion

The results of this experiment replicate the main results reported by Bouton and García-Gutiérrez (2006): Regardless of whether the rats received 16- and 4-min intervals or 4-min and 1-min intervals, Long+/Short− discrimination learning was considerably more successful than Short+/Long−. The present experiment is the first to compare the 16/4 and 4/1 conditions directly; the magnitude of the asymmetry in responding to the CS was surprisingly comparable in the two conditions. Moreover, the asymmetry was also evident in groups that received the same positive 4-min ITI (that is, Groups 4+/1− and 4+/16−). The results thus extend the findings of Bouton and García-Gutiérrez (2006), and suggest that the ITI asymmetry is replicable and robust.

Responding in extinction followed the same pattern. The extinction results are noteworthy for at least two reasons. First, they indicate that discriminative performance can occur in the absence of the US. Thus, the functional stimulus that controlled performance was the time since the last CS (i.e., the ITI). Second, unlike a report with explicit feature-positive and feature-negative discriminations (Hearst, 1987), extinction did not unveil extant learning in the difficult Short+/Long− discriminations. However, it is worth noting that previous results do suggest that Short+/Long− discriminations may be learned even when they are not expressed in behavior: Bouton and García-Gutiérrez (2006) found that initial training with a 4+/16− discrimination made it difficult to learn a subsequent 16+/4− discrimination.

In the 4/1 conditions, the Long+/Short− vs. Short+/Long− asymmetry was also apparent in responding during the pre-CS period (see also Bouton & García-Gutiérrez, 2006). Group 4+/1− used the mere passage of time in the ITI as a cue predicting the food pellet, and responded accordingly at the end of the long ITI; Group 1+/4− was slower to learn its corresponding discrimination. (Notice that the effect in the elevation scores is not an artifact of this difference in the pre-CS scores, because the elevation score subtracts pre-CS from CS responding.) Interestingly, there was no tendency to respond at the end of the positive ITIs in the 16/4 conditions, where pre-CS responding was overall considerably lower. The lack of discriminative pre-CS performance in the 16+/4− as opposed to 4+/1− condition could be interpreted as a new demonstration of the well-known fact that timing is less accurate with longer intervals (e.g., Gibbon, 1991). However, this idea may be challenged by the evidence of clear discriminative responding to the CS in the 16+/4 group, which indicates that the 16- and 4-min intervals could be used quite accurately. Indeed, discriminative responding in the CS appeared to be very similar in the 16+/4− and 4+/1− groups. Bouton and García-Gutiérrez (2006) suggested that longer intervals might also be less salient than shorter intervals, and therefore more likely to control performance to the CS as occasion setters—stimuli that enable responding to a CS rather than elicit responding directly (e.g., Holland, 1992). Holland (1989) has shown that occasion setting is especially likely to develop when the occasion setter is less salient than the target CS. Of course, it is also possible that conditioning to the ITI alone was merely below a behavioral threshold necessary to produce performance without the CS.

One feature of Bouton and García-Gutiérrez’s (2006) results was not replicated. In their comparison of the 4+/1− and 1+/4− conditions, they found, as we did here, that the discrimination (and its asymmetry) was evident in responding during the pre-CS period. However, for their Group 4+/1−, responding in the CS first increased and then decreased systematically over the course of discrimination training, as if the more informative 4-min ITI began to block excitatory conditioning of the CS (cf. Wagner, Logan, Haberlandt, & Price, 1968). It is possible that the blocking result was related to the variability of the ITI used in the previous study, or to the double alternation trial schedule used there. Alternatively, the degree of blocking by the ITI cue would presumably depend on the relative salience of the CS, with blocking by ITI occurring more readily with a less salient CS (e.g., Rescorla & Wagner, 1972). The present experiment was run in an apparatus that was different than the one used in the experiments of Bouton and García-Gutiérrez (2006). It is conceivable that CS salience was stronger in the present apparatus, reducing the strength of any blocking effect.

The main point of Experiment 1, however, is that it confirmed a robust asymmetry in temporal discrimination with 16/4 and 4/1 ITI combinations that was evident during both acquisition and extinction.

Experiment 2

The second experiment was designed to test an explanation of the asymmetry in Long+/Short− and Short+/Long− discrimination learning. During the course of a long ITI, the organism must first pass the temporal cue corresponding to the shorter ITI. This creates a bias that might explain the asymmetrical discriminative performance. As an example, consider a clock that starts when each tone terminates (and begins the next ITI) in the 16+/4− and 4+/16− procedures. For the 16+/4− condition, every time the clock reaches the 16^th minute, the next tone is reinforced. But to reach the 16^th minute, the clock must first go through Minute 4. In the 16+/4− discrimination, the 4-min cue is unambiguously associated with nonreinforcement: On negative trials, a tone is presented at minute 4, and no reinforcement occurs. On positive 16-min trials, the clock also passes through Minute 4 and is not reinforced. In contrast, in the 4+/16− discrimination, there is an inherent ambiguity in what the 4-min cue signals. On positive trials, the clock reaches Minute 4 and the tone is reinforced. But on negative trials, en route to the end of the 16-min interval, the 4-min cue is not reinforced. Thus, in the difficult 4+/16− discrimination, the 4-min readout is ambiguously associated with both reinforcement and nonreinforcement. Perhaps this difference is the source of the asymmetry in learning Long+/Short− and Short+/Long− discriminations.

Experiments 2a and 2b examined this “ambiguous short cue hypothesis” by testing the effects of adding a US at the end of the short interval embedded in each long ITI. Experiment 2a compared two groups that received the difficult 4+/16− discrimination. One received the procedure used in Experiment 1, with its ambiguous 4-min cue. The second group, however, received a noncontingent US delivered at the end of Minute 4 of every 16-min ITI. Thus, whenever the clock reached Minute 4 after a CS, a US was delivered—rendering the 4-min cue an unambiguous predictor of the US on all trials. If the asymmetry is due to ambiguity of the short cue in the Short+/Long− discrimination, the added USs should make the discrimination easier to learn.

Experiment 2b extended the analysis in two ways. First, it arranged a similar test in the 4/1 condition. That is, it compared a 1+/4− condition with a 1+/4− condition in which a US was added after the 1^st minute of each 4-min ITI. Second, it also examined the effects of adding a US at the end of minute 1 during the 4-min ITIs of the easier 4+/1− condition. In this case, adding a US after the 1^st minute of the 4-min ITI would make the 1-min cue more ambiguous than in the typical 4+/1− condition: The 1-min cue is now associated with no pellet on the 1-min trials, but a pellet during the 4-min trials. According to the ambiguous short cue hypothesis, the extra US might therefore interfere with learning the relatively easy 4+/1− discrimination. Experiment 2b also included a test in which we assessed whether the extra pellets came to control discriminative responding.

Method

Subjects and Apparatus

Experiments 2a and 2b involved 16 and 32 female rats (respectively) from the same supplier as those in Experiment 1. Both sets of rats had served in earlier experiments in which a distinctly different box in the laboratory had been paired with food pellets. The apparatus, the CS, and the US were the same as those used in Experiment 1.

Procedure

Experiment 2a

Magazine training was not necessary because of the rats’ previous treatment. There were 28 daily 82-min sessions, each consisting of four 4-min and four 16-min fixed ITIs. The rats were assigned to two groups (n = 8) that both received a 4+/16− schedule similar to that used in Experiment 1. Trials were scheduled in an RRNNRRNN sequence on odd-numbered days and NNRRNNRR on even numbered days. The “No Pellet” group received a simple 4+/16− schedule, with no extra experimental events. In contrast, the “Pellet” group received the same training with the exception that an unsignaled US (consisting of the usual two food pellets) was always presented at the end of Minute 4 of each 16-min ITI.

Experiment 2b

Magazine training once again was not necessary. Two groups (n = 8) received a 1+/4− procedure and two groups received a 4+/1− procedure like the ones in Experiment 1. One 4+/1− and one 1+/4− group received the familiar procedures with no extra experimental events. Another 4+/1− group and a 1+/4− group were each given an unsignaled US (consisting of the usual two food pellets) at the end of Minute 1 of each 4-min ITI. These groups were designated the “4+/1− Pellet” group and “1+/4− Pellet” group, respectively. There were eight reinforced and eight nonreinforced trials during each session; trials were scheduled in RRNNRRNNNNRRNNRR, NNRRNNRRRRNNRRN, RNNRNRRRNRRNRNNN, and NRRNRNNNRNNRNRRR sequences that cycled every four days.

All groups received 18 initial sessions of discrimination training. After that, the treatment of the groups given the 1+/4− and 4+/1− discriminations diverged. Rats receiving the 1+/4− discriminations received 14 additional sessions as described above, for a total of 32 daily sessions. In contrast, rats in the 4+/1− conditions received tests that were designed to determine whether the US added to the 4-min ITI provided an extra discriminative cue. On Days 19 and 20, Group 4+/1− Pellet and Group 4+/1− No Pellet underwent a “pellet test” in which they were tested with and without the unsignaled USs added to each 4-min ITI. One session contained the US at the end of Minute 1 of the 4-min ITI, and one did not; these sessions were conducted in a counterbalanced order. If the pellet had replaced the ITI as the cue signaling reinforcement in Group 4+/1− Pellet, its removal in the pellet test should disrupt performance.

Results

Experiment 2a

The results are presented in Figure 3. Although the 4+/16− No Pellet discrimination was once again learned slowly, it was eventually learned. However, adding the pellet US at the end of Minute 4 of each 16-min ITI did not facilitate that learning. If anything, the added US interfered with acquisition of the discrimination.

Figure 3.

Responding in the CS (elevation scores) and in the pre-CS period (“pre,” in no. responses) over sessions in Experiment 2a. Different groups are shown in different panels. 16 and 4 = duration of the intertrial interval (mins); +, − = tone reinforced or nonreinforced at the end of the interval, respectively.

The elevation scores were submitted to a Group × ITI × Session ANOVA. The ANOVA revealed a significant effect of ITI, F (1,14) = 7.26, MSE = 5.66, and importantly, a Group × ITI interaction, F (1,14) = 4.46, MSE = 5.66. The Session main effect, F (27, 378) = 6.25, MSE = 6.47, and the ITI × Session interaction, F (27,378) = 3.64, MSE = 1.98, were also reliable. To analyze the Group × ITI interaction, we compared the effect of ITI (collapsed over all sessions) in each group. Although there was a significant difference in Group 4+/16− No Pellet, F (1,7) = 6.76, MSE = 9.67, there was no such difference in Group 4+/16− Pellet, F (1,7) < 1, MSE = 1.27. Thus, the extra US introduced at minute 4 of the 16-min ITI reduced the modest discrimination that otherwise developed here in the 4+/16− condition.

An identical ANOVA was conducted on the pre-CS scores. Although there was a significant main effect of Session, F (27,378) = 3.26, MSE = 2.29, which was consistent with a general increase in responding with training, no other main effect or interaction approached significance.

The results clearly suggest that the extra US added at minute 4 during the negative 16-min ITI did not improve solution of this Short+/Long− discrimination.

Experiment 2b

Acquisition

The results of the 18 acquisition sessions given the 4+/1− groups and the 32 acquisition sessions given the 1+/4− groups are shown in Figure 4. As before, the 4+/1− groups learned the discrimination more quickly than the 1+/4− groups, and this difference was apparent in both the elevation scores and the pre-CS scores. Interestingly, as in Experiment 2a, there was some evidence that the Short+/Long− groups eventually learned the discrimination. But adding the US to the long ITI did not improve learning the 1+/4− discrimination; the extra USs interfered with both discriminations.

Figure 4.

Responding in the CS (elevation scores) and in the pre-CS period (“pre,” in no. responses) over sessions in Experiment 2b. Different groups are shown in different panels. 4 and 1 = duration of the intertrial interval (mins); +, − = tone reinforced or nonreinforced at the end of the interval, respectively.

The elevation scores during the first 18 acquisition sessions (which all groups received) were submitted to a Discrimination Type (Short+ or Long+) × ITI (reinforced vs nonreinforced) × Added Pellet × Session ANOVA. There were significant main effects of both ITI, F (1,28) = 65.95, MSE = 5.06, and Session, F (17,476) = 14.78, MSE = 5.39. There was also an ITI × Session interaction, F (17,476) = 10.24, MSE = 1.41. Furthermore, the discrimination was again stronger in the Long+/Short− groups, as indicated by a reliable Discrimination Type × ITI interaction, F (1,28) = 50.31, MSE = 5.06. The effect of ITI also depended on whether a US was added or not, as indicated by an ITI × Added Pellet interaction, F (1,28) = 5.72, MSE = 5.06. In either the 1+/4− or the 4+/1− conditions, the added pellet appeared to reduce the effect of ITI. Nonetheless, simple effect tests revealed a reliable ITI effect in each of the 4+/1− groups, Fs (1,7) ≥ 36.54, MSE = 3.91, but neither of the 1+/4− groups, Fs (1,7) < 1, MSEs ≤ 5.81. The effect of the added pellet seemed to be especially pronounced in the 4+/1− discrimination, as suggested by an ITI × Added Pellet × Discrimination Type interaction, F (1,28) = 4.85, MSE = 5.06. Confirming this, separate ANOVAs comparing the two groups that received the 4+/1− and the 1+/4− discriminations revealed a significant Pellet × ITI interaction in the 4+/1− groups, F (1,14) = 8.42, MSE = 6.34, but not in the 1+/4− groups, F 1,14) < 1, MSE = 7.80.

Parallel ANOVAs on the pre-CS scores revealed the following. In the overall Discrimination Type (Short+ or Long+) × ITI (reinforced vs nonreinforced) × Added Pellet × Session ANOVA (over the common first 18 sessions), there were effects of ITI, F (1,28) = 94.88, MSE = 1.27, Session, F (17,476) = 3.64, MSE = 1.36, and an ITI × Session interaction, F (17,476) = 3.86, MSE = 0.52. Further, consistent with the asymmetry evident in pre-CS responding in Experiment 1, there was a Discrimination Type × ITI interaction, F (1,28) = 39.88, MSE = 1.27, as well as a Discrimination Type × Session interaction, F (17,476) = 3.86, MSE = 1.36, and a Discrimination Type × ITI × Session interaction, F (17,476) = 2.73, MSE = 0.52. Regarding the pellet variable, neither its main effect nor any interaction involving it was reliable except for the Discrimination Type × Pellet interaction, F (1,28) = 6.31, MSE = 12.18. Groups 4+/1−, 4+/1− Pellet, and 1+/4− responded more on reinforced than nonreinforced trials, Fs (1,7) ≥ 12.54, MSE = 0.57; Group 1+/4− Pellet did not, F (1,7) = 2.08, MSE = 0.68. Once again, if there was an effect of adding the US, it was thus to interfere modestly with discrimination learning. Separate ANOVAs on pre-CS responding in the 4+/1− and 1+/4− groups revealed no interactions with the Pellet variable.

Pellet test

It is possible that adding the US in the long ITIs changed what the rats learned. That is, it is conceivable that the addition of the pellet in the 4-min ITI allowed the 4+/1− pellet group to respond differentially without learning anything about the ITI. To test this possibility, the groups in the 4+/1− condition underwent testing in which free USs were present or absent at the end of Minute 1 during the 4-min ITIs. The results of these tests are shown in Figure 5. As the figure suggests, removal of the extra USs had no apparent impact on the discrimination in the rats that had received them during training. A Test Pellet × Acquisition Pellet × ITI × Test Order interaction revealed only a main effect of ITI, F (1,12) = 94.58, MSE = 2.30, and a main effect of the presence or absence of the pellet during the test, F (1,12) = 6.15, MSE = 2.27. No other effects or interactions approached significance. Although both groups responded somewhat more during sessions that included the extra pellets, the pellet did not influence discriminative performance. Thus, the rats in Group 4+/1− Pellet appeared to use the ITI, rather than the presence of the added pellets, as the cue to respond to the CS on the next trial.

Figure 5.

Responding in the CS (elevation scores) during the pellet test of Experiment 2b. Conditioning treatment (group labels) are designated by panel labels. Test treatments are indicated on the x-axis.

Responding in the pre-CS periods was analyzed with an identical ANOVA. This analysis once again revealed a significant ITI main effect, F (1,12) = 29.72, MSE = 2.09, but no other reliable effect or interaction. For animals in the 4+/1− condition, the passage of time in the ITI, rather than a US occurring during the long ITI, was the cue that controlled responding in both the CS and the pre-CS period.

Discussion

The results of these experiments suggest that adding a US at the end of the short interval embedded in each long interval does not improve learning the Short+/Long− discriminations (4+/16− in Experiment 2a and 1+/4− in Experiment 2b). Thus, reducing the “ambiguity” of the short cue did not improve performance. Indeed, the extra pellets hurt, rather than facilitated, the development of both Short+/Long− discriminations. The results thus suggest that the inherent ambiguity of the short cue embedded in each long ITI is not the source of the asymmetry that is of interest here. The fact that the extra USs also modestly interfered with the 4+/1− discrimination (Experiment 2b) is consistent with the possibility that adding the extra event to the ITIs made timing in the ITI more difficult.

Although the extra USs had an impact on behavior, the results of the pellet test given the 4+/1− groups in Experiment 2b suggest that they did not qualitatively change the solution of the discrimination. Indeed, the fact that removing the extra USs in Group 4+/1− Pellet did not change their differential performance in 4-min and 1-min ITIs suggests that the rats did not use the pellet as a discriminative cue. This result continues to suggest that ITIs in the range of 1–4 minutes are salient discriminative cues—they still dominate the control of behavior when a cue provided by a 2-pellet US is also available.

Experiment 3

Experiment 3 was designed to test another account of the asymmetry in the ITI discriminations evident in Experiments 1 and 2b. It is possible that a Short+/Long− discrimination is difficult to learn because a CS presented after a long ITI might serve as a sign that a reinforcer is coming on the horizon. The upper part of Figure 6 illustrates the point by showing 4+/16− and 16+/4− trials in a double-alternating (NNRR) procedure. It can be seen that, for Group 4+/16−, CS presentations at the end of the long ITIs might be in a position to signal that a reinforcer is coming soon. If this caused some anticipation of the reinforcer in the negative trials, it might make it difficult to observe good discriminative performance in a Short+/Long− procedure.

Figure 6.

Top: CS and US presentations in the 4+/16− and 16+/4− discrimination procedures. Bottom: CS and US presentations in the 4+/16− and 16+/4− discriminations with extra USs (pellets) added to modify the reduction in delay to reinforcement signaled by the nonreinforced trials.

The intuition can be made more precise by application of Delay Reduction Theory (e.g., Fantino, 1969; Fantino, Preston, & Dunn, 1993), a theory of conditioned reinforcement which holds that a CS’s conditioned reinforcing value depends on its ability to signal a reduction in the delay to the next reinforcer. According to the theory, the value of a conditioned reinforcer is a function of how much the onset of a stimulus reduces the expected time to reinforcement relative to the overall time to reinforcement without the stimulus (e.g., Williams, 1994). It is thus instructive to compare the delay reduction present in the negative trials of the 4+/16− and the 16+/4− procedures. Although “overall time to reinforcement” can be interpreted as the global average time between reinforcers, there is evidence that the overall time to reinforcement is estimated locally rather than globally (e.g., Goldschmidt, Lattall, & Fantino, 1998). We therefore begin our discussion by considering a local estimate of the overall reinforcement time (a global estimate is also considered below). In a double-alternated 4+/16− discrimination, which we consider for its simplicity, the local time between reinforcers when two successive N trials occur is 36 minutes (each of the two N trials involve a 16 minute ITI, and the next reinforcer is presented after the next positive 4-min ITI). On the first of the two nonreinforced trials, the time between the tone and the next reinforcer is 20 minutes (the next 16-min and 4-min ITIs). Thus, the proportional reduction in time to the reinforcer signaled by the first tone is (36 - 20)/36 = 0.44. On the second N trial, the time between the tone and the next reinforcer is 4 minutes (the next 4-min ITI), with a consequent delay reduction of (36 - 4)/36 = 0.89. Therefore, in the 4+/16− discrimination, the average delay reduction signaled by a tone after each negative ITI is 0.66. This value is considerably higher than that in the easier 16+/4− procedure. In the double alternating 16+/4− procedure, the interval between successive reinforcers when nonreinforced signals occur is 24 minutes (two negative 4-min ITIs followed by a positive 16-min ITI). The delay to reinforcement after the first nonreinforced tone is 20 minutes (the next negative 4-min ITI and the subsequent positive 16-min ITI), with a delay reduction of (24 - 20)/24 = 0.17. The delay to reinforcement between the second nonreinforced tone and the next reinforcer is 16 minutes (the next 16-min ITI), with a delay reduction of (24 - 16)/24 = 0.33. Therefore, in the 16+/4− discrimination, the average delay reduction signaled in the negative ITI is thus 0.25—a considerably lower value than that in the 4+/16− ITI (0.66). There might therefore be less incentive to respond inappropriately in the negative trials in the easier 16+/4− procedure.

Using such calculations as a guide, Experiment 3 added USs at selected points during some of the intertrial intervals to modify the reduction in delay to reinforcement that was signaled by N trials. Two groups received the double-alternated 16+/4− discrimination, and two groups received the corresponding 4+/16− discrimination. In each condition, one group received pellet USs added at specific points that were designed to equate the delay reduction signaled by the nonreinforced CS (bottom half of Figure 6). In the 4+/16− Pellet group, a US was added at the end of the 12^th minute of the first of the two successive N (16-min ITI) trials. This caused the interval between successive reinforcers when nonreinforced CSs occurred to be 24 minutes (the remaining 4 min of the first 16-min trial, the next 16-min trial, and then the next 4-min positive ITI). The delay to reinforcement following the first nonreinforced tone was 20 minutes, for a delay reduction of (24 - 20)/24 = 0.17. The delay to reinforcement following the second nonreinforced tone was 4 minutes, for a delay reduction of (24 - 4)/24 = 0.83. The average delay reduction signaled by a nonreinforced tone in this group was thus 0.50.

The 16+/4− group that received added USs also received an added US in the first of two 16-min ITIs. Specifically, these animals received an added US at the end of the 4^th minute during the first of the pair of 16-min ITIs. For this group, the interval between successive reinforcers when nonreinforced CSs occurred was thus 12 minutes (two successive 4-min trials plus the 4 minutes after which the pellet was delivered in the 16-min trial). The delay to reinforcement following the first nonreinforced tone was eight minutes (the next 4-min N trial plus the 4 min in the 16-min positive trial), for a delay reduction of (12 - 8)/12 = 0.33. The delay to reinforcement following the second nonreinforced tone was 4 minutes, for a delay reduction of (12 - 4)/12 = 0.67. The average delay reduction signaled by a nonreinforced tone in this group was thus 0.50--exactly the same as in the 4+/16− Pellet group. Thus, if the asymmetry apparent in the previous Long+/Short− and Short+/Long− discriminations were due to differential delay reduction in the nonreinforced trials, then a comparison of the two groups with added USs should fail to reveal the asymmetry.

As noted earlier, the preceding calculations of delay reduction used the “local” time between the reinforcers surrounding the N trials as the background time to reinforcement (see Goldschmidt et al., 1998). Analogous calculations with the session’s overall average time between reinforcers (20 min for the No Pellet and 13.3 min for the Pellet groups), a more “global” estimate of the background time to reinforcement, yielded an average proportional delay reduction of 0.40 and 0.10 for Groups 4+/16− and 16+/4−, respectively, and 0.10 and 0.55 for Groups 4+/16− Pellet and 16+/4− Pellet, respectively. By these calculations, then, the added pellets actually reversed the delay reduction signaled by the N trials and should theoretically make the discrimination more difficult in Group 16+/4− Pellet than Group 4+/16− Pellet.

Method

Subjects and Apparatus

The subjects were 32 female Wistar rats obtained from the same supplier and housed and maintained as in the previous experiments. They had previously been in an unrelated experiment like the one that preceded Experiments 2a and 2b. The apparatus, CS, and US were the same.

Procedure

Two groups each received the 16+/4− and 4+/16− of the earlier experiments. In each of 15 daily sessions of discrimination training, there were a total of 12 trials, of which 6 were reinforced and 6 were nonreinforced. These were scheduled such that on even-numbered sessions, an RRNNRRNNRRNN was in effect, and on odd-numbered sessions an NNRRNNRRNNRR was in effect. In two groups (one from each condition), USs were also presented during the ITIs themselves so that delay reduction was manipulated. Group 4+/16− Pellet received the two-pellet US 12 min into the first of the two successive 16-min ITIs. Group 16+/4− Pellet was given the extra two-pellet US 4 min into the first of the two successive 16-min ITIs. As explained in the introduction, the Pellet groups gave us 4+/16− and 16+/4− groups for which delay reduction signaled by the N trials was either equivalent (by a local calculation of time to the next reinforcer) or lower in the 4+/16− group (by a global calculation of average time to the next reinforcer).

Results

The results are presented in Figure 7. As the figure suggests, rats given the 16+/4− procedure once again discriminated the positive and negative trials more rapidly than animals given 4+/16−. Most important, adding USs to selected ITIs did not reduce the asymmetry.

Figure 7.

Responding in the CS (elevation scores) and in the pre-CS period (“pre,” in no. responses) over sessions in Experiment 3. Different groups are shown in different panels. 16, 4, and 1 = duration of the intertrial interval (mins); +, − = tone reinforced or nonreinforced at the end of the interval, respectively; Pellet refers to the extra US added during intertrial intervals in two of the groups.

The elevation scores were submitted to a Discrimination Type (Short+ or Long+) × ITI (reinforced vs nonreinforced) × Added Pellet × Session ANOVA. The analysis revealed main effects of ITI, F (1,28) = 25.67, MSE = 8.50, Session, F (14, 392) = 9.17, MSE = 3.74, and an ITI × Session interaction, F (14,392) = 8.50, MSE = 1.14. Confirming the presence of the usual asymmetry, there was a Discrimination Type × ITI interaction, F (1,28) = 19.80, MSE = 8.50. Simple effect tests (collapsing over sessions) indicated a reliable effect of ITI in the Long+/Short− groups, F (1,15) = 26.22, MSE = 26.22, MSE = 14.68, but not in the Short+/Long− groups, F (1,15) < 1, MSE = 3.45. The Discrimination Type × ITI × Session interaction, F (14,392) = 2.63, MSE = 1.14, and the Discrimination Type × Session interaction, F (14,392) = 2.53, MSE = 3.74, were also reliable. No effect or interaction involving the pellet factor was significant, although the ITI × Pellet interaction fell just short of the conventional rejection criterion, F (1,28) = 3.97, MSE = 8.50, p = .06. The figure suggests that the near-interaction was due to modestly weaker discriminations in both Pellet groups.

It is clear that the pellet manipulation did not eliminate the difference between the 16+/4− and 4+/16− conditions. As a further check, we analyzed the two Pellet groups with a separate Discrimination Type × ITI × Session ANOVA. The ANOVA revealed a reliable main effect of ITI, F (1,14) = 5.20, MSE = 7.72, and Session, F (14,196) = 3.81, MSE = 3.64, as well as an ITI × Session interaction, F (14,196) = 2.74, MSE = 1.09. Most important, the asymmetry was clearly evident, as indicated by a reliable Discrimination Type × ITI interaction, F (1,14) = 11.54, MSE = 7.72, as well as a Discrimination Type × ITI × Session interaction, F (14,196) = 1.89, MSE = 1.09. No other interactions were significant.

An overall ANOVA on the pre-CS scores revealed significant main effect of ITI, F (1,28) = 11.70, MSE = 0.72, and Session, F (14,392) = 1.73, MSE = 0.68. There was also a reliable interaction between ITI and Discrimination Type, F (1,28) = 11.19, MSE = 0.72. As the figure suggests, a discrimination between positive and negative trials was mainly evident in the 16+/4− Pellet group. This pattern was also revealed in a separate ANOVA on the two Pellet groups, which uncovered reliable effects of ITI, F (1,14) = 13.56, MSE = 0.59, Session, F (14,196) = 1.85, MSE = 0.76, and an ITI × Discrimination Type interaction, F (1,14) = 16.16, MSE = 0.59. It seems possible that the extra USs, which were delivered during half of the positive 16-min ITIs in this group, caused higher responding during the corresponding pre-CS periods.

Discussion

The results indicate that the asymmetry between Long+/Short− and Short+/Long− learning is maintained after steps are taken to reduce (or reverse) the difference in the delay reduction signaled on the nonreinforced trials. In the present experiment, we added extra USs to selected intertrial intervals so that (1.) when overall reinforcement rate was estimated locally (e.g., Goldschmidt et al., 1998), N trials in both the 16+/4− Pellet and 4+/16− Pellet condition signaled an equivalent reduction in delay to reinforcement and (2.) when overall reinforcement rate was estimated globally, N trials in the 16+/4− Pellet condition signaled a greater delay reduction than N trials in the 4+/16− Pellet condition, which would theoretically make the 16+/4− Pellet discrimination more difficult. However, despite the addition of the extra USs, the original asymmetry was intact; Group 16+/4− Pellet acquired discriminative performance more rapidly than Group 4+/16− Pellet. These results strongly suggest that the asymmetry in Long+/Short− vs. Short+/Long− discrimination learning is not a result of differential delay reduction signaled in the nonreinforced trials.

There are other reasons to question the “better times are ahead” hypothesis as an explanation of the poor learning of Short+/Long− discriminations. First, Short+/Long− procedures do not merely result in unusually high performance in the presence of the CS on the long negative trials. As documented in Experiment 1, poor performance in the Short+/Long− discriminations is a consequence of both higher responding on the negative trials and lower responding in the positive trials; lower responding on the positive trials would require an additional explanation. Second, it is worth noting that this application of delay reduction theory is unique; the theory has usually been tested in chained schedules of reinforcement, where differences in the delay reduction signaled by a conditioned reinforcer is reflected in choice behavior. To our knowledge, few experiments from the delay reduction literature show that a cue signaling more delay reduction actually elicits more conditional responding. An exception is a study by Fantino (1982) in which pigeons responded more to a keylight associated with a fixed-time 30-s schedule of reinforcement when the ITI (the initial link in the chained schedule) that preceded it was 120 s rather than 10 s. Although the results are consistent with the idea that responding is higher with a CS that signals a greater reduction in time to reinforcement, the experimental conditions were very different from the present ones, where the CSs were actually nonreinforced--the reinforcers they potentially predicted occurred more remotely in time. It is also possible that the suppressed performance after the short ITI in Fantino’s (1982) study was alternatively due to priming of the CS and/or US in a refractory state in short− term memory from the preceding trial (Wagner, 1981; see Sunsay, Stetson, & Bouton, 2004).

In summary, the results of this experiment provide no support for the idea that the discriminative asymmetry is due to N trials signaling “better times are ahead,” and perhaps inflating responding on the N trials, in the Short+/Long− procedure.

Experiment 4

Having found no evidence to support the “ambiguous short− cue hypothesis” or the “things are getting better hypothesis,” it was important to consider the temporal elements hypothesis described in the Introduction in more detail. That hypothesis holds that the passage of time in the ITI may involve a sequence of hypothetical elements (A-B-C…). As noted in the introduction, the conceptualization of Desmond and Moore (1988) envisions temporal elements as sequential and overlapping, so that a short ITI might end at stimulus A, and a longer interval might end at AB (B is added to A). According to this analysis, Long+/Short− learning might be more successful than Short+/Long− because feature-positive discriminations (AB+/A−) are learned more quickly than feature-negative discriminations (A+/AB−). Superior discrimination learning with the AB+/A− procedure is consistent with elemental models of compound conditioning (e.g., Rescorla & Wagner, 1972), although it is worth noting that configural models like that of Pearce (1987, 1994) do not predict such an effect.

A further challenge for the temporal elements hypothesis is that although the feature-positive effect has been documented in conditioning procedures in which A and B have been presented simultaneously or when B precedes A (e.g., Hearst, 1978, 1984), we are not aware of any demonstrations of the effect in procedures in which the onset of B occurs after the onset of A, the state of affairs required by the temporal elements hypothesis. This difference is potentially important, because subtle differences in the temporal sequencing of CSs can have profound effects on the mechanisms animals use to solve discriminations in compound conditioning (as suggested by the occasion-setting literature, e.g., Holland, 1992; Swartzentruber, 1995). The purpose of Experiment 4 was therefore to test for a feature-positive effect in a method in which explicit CSs were substituted for the temporal elements A and B, and the CSs were specifically arranged so that presentation of the feature (B) followed the onset of A.

Two groups received either AB+/A− or A+/AB− training in a compound conditioning procedure. Noise and light CSs (counterbalanced) played the roles of A and B. As in the conceptualization of Desmond and Moore (1988), the arrangement of A and B was sequential and overlapping. For both groups, A was always 20 s in duration. On compound (AB) trials, B was presented for 10 s during the final 10 s of A. This arrangement further guaranteed that the presence or absence of B, and not the duration of A or the total trial duration, was the only cue that differentiated reinforced and nonreinforced trials. For simplicity, the procedure restricted itself to A and B and did not include a third stimulus analogous to the tone CS used in the previous experiments. The procedure thus modeled the direct effect of time on responding in the present 4+/1− versus 1+/4− discriminations, where time in the ITI elicited magazine entry behavior in the pre-CS period, i.e., before a CS was presented (Experiments 1 and 2b; Bouton & Garcia-Gutierrez, 2006). If the asymmetry in pre-CS responding in those conditions was due to a feature-positive effect enabled by hypothetical A and B temporal elements, then we should expect a similar effect in the present experiment. Failure to observe a feature-positive effect in the current preparation would make the temporal elements hypothesis a less plausible account of the present results.

Method

Subjects and apparatus

The subjects were 16 female Wistar rats from the same supplier and housed and maintained like the rats in the previous experiments. They were approximately 110 days old at the start and had previously participated in a fear conditioning experiment conducted in a separate apparatus with a very different CS (darkness provided by offset of a houselight).

The apparatus was the same as in the preceding experiments. Two new counterbalanced CSs were used. One CS was an intermittent white noise (pulsed 4 times/s) presented through the speaker mounted to the ceiling of the sound attenuation chamber. The clicker was 70 dB(A) above a 65–66 dB(A) background. The other CS was a flashing (0.4-s on alternated with 0.1-s off) of the 28-V panel light that was mounted on the wall above and to the left of the food cup. As usual, the US was two 45-mg food pellets.

Procedure

The first daily session was a magazine training session in which the rats were placed in a chamber and given 30 pellets distributed over 20 min. The experiment then occurred in daily sessions conducted over the next three days. Half the trials of each session consisted of a 20-s presentation of CS A alone (noise or light, counterbalanced), and half the trials consisted of an AB compound in which CS B (light or noise, counterbalanced) was presented during the last 10 s of the 20-s A. For Group FP, the AB trials ended in the US and the A trials did not. For Group FN, the A trials ended in the US and the AB trials did not. Each session contained 16 reinforced and 16 nonreinforced trials separated by a variable ITI of 2 min (± 30 s). Trials types were double-alternated, and each session started with a reinforced trial.

Data analysis focused on the second 10 s of each 20-s trial, i.e., the segment when CS B was specifically present or absent. Elevation scores were calculated in the usual way by subtracting responses made during the 10-s period before onset of A from the responses in this 10-s CS period.

Results and Discussion

Figure 8 summarizes the results over the six 8-trial blocks of the experiment. It is clear that Group FP readily learned the discrimination, and that Group FN did not learn it within the number of trials administered here. The pattern was confirmed by a Group (FP vs. FN) × Feature Stimulus (noise vs. light B stimulus) × Trial-type (R vs. N) × Block ANOVA. The analysis revealed a reliable Group main effect, F (1,12) = 13.83, MSE = 5.44, as well as Group × Trial-type, F (1,12) = 14.20, MSE = 3.19, and Group × Trial-type × Block, F (5,60) = 3.00, MSE = 0.77, interactions. Simple effects exploring the Group × Trial-type interaction confirmed reliably more responding on reinforced than nonreinforced trials in Group FP, F (1,7) = 8.48, MSE = 9.71, but not in Group FN, F < 1, MSE = 0.82. The Block effect and the Group × Block interaction were also significant, Fs(5,60) ≥ 3.96, MSE = 1.12, as was the Trial-type × Block interaction, F (5,60) = 3.72, MSE = 0.77, and the Trial-type main effect, F (1,12) = 11.66, MSE = 3.19. The noise CS generally elicited more responding than the flashing light CS, as suggested by a reliable Feature Stimulus effect, F (1,12) = 6.71, MSE = 5.44, a Group × Feature Stimulus interaction, F (1,12) = 5.38, MSE = 5.44, a Group × Feature Stimulus × Trial-type interaction, F (1,12) = 9.06, MSE = 3.19. However, the feature-positive effect was evident regardless of which CS played the role of CS B; for example, during the last two 8-trial blocks (the last session), there was a significant Group (FP vs. FN) × Trial-type interaction in the subgroups that had the click or the light in the role of CS B, Fs (1,6) ≥ 5.99, MSEs ≤ 6.53. None of the effects not mentioned above approached significance.

Figure 8.

Responding in the CS (elevation scores) over 8-trial blocks of Experiment 4. FP = feature positive, FN = feature negative; + and − = reinforced trials and nonreinforced trials, respectively.

An identical ANOVA on responding during the pre-CS period revealed no main effects or interactions. The mean responding during the pre-CS period was 1.16 and 1.26 for the R and N trials of Group FP and 1.30 and 1.39 for the corresponding trials of Group FN.

The results clearly suggest that a feature-positive effect occurs in the present conditioning preparation, and when the onset of stimulus B follows that of stimulus A, as hypothetical temporal elements would (e.g., Desmond & Moore, 1988). A subsequent experiment found the same results when the duration of A and B were both 10 s and the AB compound involved their simultaneous onset and offset (Bouton & Doyle-Burr, in preparation).

General Discussion

The present experiments replicate and extend the asymmetry in temporal discrimination learning reported by Bouton and García-Gutiérrez (2006). The difference between Long+/Short− and Short+/Long− learning was observed in several experiments (Experiments 1, 2b, and 3). The results of Experiment 1 suggested several important conclusions. First, the asymmetry in discriminative responding was equally evident with two sets of intertrial intervals (16 vs. 4 min and 4 vs. 1 min). Second, the asymmetry in responding persisted when the US was omitted in extinction, suggesting that the effective stimulus controlling the asymmetry is the interval of time since the last CS rather than the last US. Third, the asymmetry was also present in pre-CS responding in the 4/1 combination, though not 16/4. As noted before, this pattern may be consistent with the idea (Bouton & Garcia-Guttierez, 2006) that short intervals are more salient than long ones, and therefore play the role of CS, directly eliciting behavior. At longer intervals, the ITI functioned more like an occasion setter; the animals did not respond to it directly, but used ITI information to disambiguate the meaning of the CS. The idea that occasion setting is more likely to develop with less salient stimuli is consistent with research on occasion setting (Holland, 1989). Overall, the results of Experiment 1 found a clear and marked asymmetry under several different conditions (16/4 discriminations, 4/1 discriminations, tone responding, pre-CS responding, acquisition, and extinction).

The experiments also tested implications of three explanations of the asymmetry effect. Experiment 2 tested the ambiguous short cue hypothesis, which notes that the organism must go through a short interval on the way to the end of a longer one. Consequently, in the easy Long+/Short− discrimination, the rat consistently receives no US whenever it encounters the short cue. In contrast, in the difficult Short+/Long− discrimination, the short cue is reinforced on positive trials but nonreinforced when it is encountered en route to the conclusion of the longer ITI. Experiment 2 tested the role of the ambiguous short cue by adding USs at the time of the short cue during the long ITI (4 min in 4+/16− and 1 min at 1+/4−). Although this should have made the short cue less ambiguous, it did not facilitate learning the Short+/Long− discrimination. It is also notable that, although adding the US to the procedure interfered to some extent with discriminative performance, it did not appear to reduce the rat’s learning about the ITI, as indicated by the results of the pellet test (Figure 5).

Experiment 3 tested an implication of the idea that the CSs at the end of long ITI in the difficult Short+/Long− paradoxically signal a substantial reduction in delay to the next reinforcer, and that this might make it difficult to treat the end of the Long ITI (or the CS presentation at the end of the long ITI) as a negative cue. Using Delay Reduction Theory as a guide, we presented extra USs at selected time points that were designed to create a delay reduction signaled by the negative trials that was either equivalent across 16+/4− and 4+/16− procedures (when overall reinforcement rate was defined locally), or greater for the 16+/4− (when overall reinforcement rate was defined globally). This manipulation did not reduce the size of the asymmetry between Long+/Short− and Short+/Long− learning. Further, we also noted that differential delay reduction cannot provide a complete account of the asymmetry, because the results of Experiment 1 indicate that Short+/Long− groups show both higher responding in the negative trials, which is implied by the theory, and lower responding in the positive trials, which is not. It is also worth noting that rats given a pseudodiscrimination in which Long and Short ITIs were each associated with reinforced trials half the time showed equivalent responding after the Short and Long ITIs (Bouton & García-Gutiérrez, 2006, Experiments 3 and 4). Thus, there is also no unconditional bias of rats to respond more to the CS after long than short ITIs.

The results also do not appear to be anticipated by the major models of interval timing, which commonly assume that animals associate reinforcers with temporal cues represented either by the number of pulses that collect in the accumulator of a pacemaker-accumulator mechanism (e.g., Gibbon et al., 1984), the behavioral state in a sequence of behavioral states (e.g., Killeen & Fetterman, 1988; Machado, 1997), or the read-out of an array of oscillators that cycle through different states with different periods (e.g., Church & Broadbent, 1990). They are also not immediately anticipated by the view that timed “packets” of responding occur as a function of the expected time to the US (Kirkpatrick, 2002). These models generally do not incorporate associative learning principles that might be ready to explain the basic findings. The temporal elements view assumes that the passage of time involves exposure to a series of hypothetical elements (e.g., A then B then C), and crucially, that animals learn about these elements in the same way they learn about explicit CSs in compound conditioning experiments. As we have noted before, this view suggests that Long+/Short− is superior to Short+/Long− because feature-positive discriminations (AB+/A−) are learned more rapidly than feature-negative discriminations (A+/AB−). Consistent with this possibility, Experiment 4 demonstrated such a “feature-positive effect” when two CSs played the role of A and B in a sequential and overlapping arrangement that was meant to capture one version of the temporal elements’ possible relationship in time (e.g., Desmond & Moore, 1988). Experiment 4’s results were especially and most directly analogous to the asymmetry evident in pre-CS responding in the 4+/1− and 1+/4− procedures (e.g., Experiments 1 and 2b), where time in the ITI elicited foodcup behavior directly and most differentially in the Long+/Short− (AB+/A−) condition (see also Bouton & Garcia-Gutierrez, 2006).

The feature-positive effect is consistent with elemental models of compound conditioning. For example, the Rescorla-Wagner model (e.g., Rescorla & Wagner, 1972) predicts that elements A and B would both acquire some associative strength on the reinforced trials in the AB+/A− procedure, whereas only A would acquire associative strength during reinforced trials in the A+/AB− procedure. Consequently, on early trials, nearly equivalent increases to both A and B would allow nearly twice the excitation to accrue to AB after AB+ than to A after A+. This difference, coupled with the fact that (1.) there would be less generalization of responding from AB+ to A in the feature-positive procedure than A+ to AB in the feature-negative procedure, and (2.) feature-negative discriminations require inhibition to develop to B, which cannot occur until some excitation has accrued to A, allows the Rescorla-Wagner model to predict the feature-positive effect. Thus, the temporal elements perspective coupled with familiar assumptions about compound conditioning might go some distance in explaining the asymmetry in temporal discrimination learning. Although the behavioral theory of timing (Killeen & Fetterman, 1988; see also Machado, 1997) also envisions reinforcement being associated with elements that change over time in a regular A-B-C sequence (in this case the elements are behavioral states), the approach does not employ a competitive learning rule like that in the Rescorla-Wagner model (see Machado, 1997, p. 243). It would not predict the present asymmetry or the feature-negative learning that occurs when A and B overlap the way they did in Experiment 4. For more discussion of the possible importance of a competitive learning rule in understanding timing, see Vogel et al. (2003).

Although a formal model of timing using temporal elements is beyond the scope of this article, the current findings begin to provide a base from which to build one. The results already suggest at least two important challenges or constraints. First, such a model will need to accommodate the finding that adding reinforcers to the ITI had relatively little impact on the asymmetry that was evident between Long+/Short− and Short+/Long− discriminations in Experiments 2 and 3. Second, such a model will also need to be explicit about how the tone CS (e.g., Stimulus C) combines with the temporal elements in the ITI (e.g., A and B) to control performance. In the current experiments, the asymmetry was evident in responding during A and AB themselves (i.e., in the pre-CS periods at the ends of ITIs in the 4+/1− and 1+/4− conditions), but it was also generally evident in responding during the CS (in C after AB vs. A in both the 4/1 and 16/4 conditions). It is worth noting that the Rescorla-Wagner model does in fact predict the feature-positive effect when a third stimulus is added to the A and AB trials in the FP and FN discriminations (i.e., an ABC+/AC− discrimination will be learned more rapidly than AC+/ABC−). However, a full model will also need to accommodate the possibility that time can work as an occasion setter with longer intervals (16/4) as well as an eliciting stimulus with the shorter intervals (4/1). As noted above, the overall pattern may be consistent with the possibility that early temporal elements (corresponding to shorter intervals) might be more salient than later temporal elements (corresponding to longer intervals).

In summary, the present results confirm that the time between trials can serve as an effective contextual cue. They also confirm an asymmetry in temporal discrimination learning that does not appear to have been anticipated by many theories of interval timing. The results may be most consistent with the view that passage of time is made up of a series of elements that acquire associative strength according to established rules of compound conditioning. Perhaps the most interesting implication is that there may be more in common between temporal discrimination learning and other types of discrimination learning than has often been supposed.