The Effects Of Interval Duration On Temporal Tracking And Alternation Learning

Abstract

On cyclic-interval reinforcement schedules, animals typically show a postreinforcement pause that is a function of the immediately preceding time interval (temporal tracking). Animals, however, do not track single-alternation schedules—when two different intervals are presented in strict alternation on successive trials. In this experiment, pigeons were first trained with a cyclic schedule consisting of alternating blocks of 12 short intervals (5 s or 30 s) and 12 long intervals (180 s), followed by three different single-alternation interval schedules: (a) 30 s and 180 s, (b) 5 s and 180 s, and (c) 5 s and 30 s. Pigeons tracked both schedules with alternating blocks of 12 intervals. With the single-alternation schedules, when the short interval duration was 5 s, regardless of the duration of the longer interval, pigeons learned the alternation pattern, and their pause anticipated the upcoming interval. When the shorter interval was 30 s, even when the ratio of short to long intervals was kept at 6:1, pigeons did not initially show anticipatory pausing—a violation of the principle of timescale invariance.

In their natural environments, animals deal with a bewildering array of complex stimuli, potential responses, other creatures, imminent dangers, and all-too-infrequent reinforcers. The relation between these myriad events typically is complicated and often unpredictable but, on occasion, simple regularities do occur. Some events occur sequentially or repeatedly, whereas others recur at temporally regular intervals. For example, if an animal lives near two equidistant foraging patches, it might do well by alternating between the two on consecutive foraging runs to avoid depleting either food source. Animals that can learn these sorts of patterns and adjust their behavior accordingly have an obvious selective advantage over those that cannot.

Nearly a century of research has started to delineate the temporal and sequential regularities to which different animals can and do respond (Ferster & Skinner, 1957; Gallistel, 1990; Richelle & Lejeune, 1980). Since fixed-interval (FI) schedules were first explored in detail by Ferster and Skinner, research on temporal control in animals has proceeded in two main directions: psychophysical and dynamic. The psychophysical approach focused on the Weber-law property of timing—that timing accuracy is approximately proportional to the interval to be timed. Expressed in terms of the proportionality between standard deviations of steady-state dependent timing measures and their means, this property has been termed scalar timing (Gibbon, 1977, 1991). In the form of the broader assertion that only relative times matter, this finding has been termed the principle of timescale invariance (Gallistel & Gibbon, 2000). Recently, the scope of this generalization has been expanded beyond asymptotic behaviors to include the acquisition of learned behavior. It is this latter assertion about timescale invariance that our set of experiments seeks to evaluate.

In contrast to the psychophysically rooted approach, the dynamic approach developed from an early finding that animals could adjust their postreinforcement pause (PRP) to a cyclically changing series of interreinforcement intervals—a behavior termed temporal tracking (Innis & Staddon, 1971). For example, animals presented with a repeating cycle of an ascending and descending arithmetic progression of intervals (8 s, 12 s, 16 s, 20 s, 24 s, 28 s, 32 s, 28 s, 24 s, 20 s, 16 s, 12 s, 8 s, and so on) will pause longer after each longer interval on the ascending leg and then shorter after each shorter interval on the descending leg. Subsequently, it was shown that under many conditions, pigeons and rats adjust their PRP immediately, reacting to the immediately preceding interval (i.e., with a lag of one interval), in a series of changing intervals (Higa, Thaw, & Staddon, 1993; Innis, 1981; Innis & Staddon, 1971). Moreover, the process seemed to be obligatory in the sense that animals would continue to behave in this way even if the result was a drastic drop in the rate of reinforcement (Wynne & Staddon, 1988), or a maladaptive short pause in a predictably long interval (Higa, Wynne, & Staddon, 1991).

Square-wave (SW) interval schedules are a subclass of these cyclic-interval schedules, with only two different interval durations (Innis & Staddon, 1970; Ludvig & Staddon, 2004; Staddon 1967, 1969). With a standard SW schedule, each of the two interval durations is repeated several times in succession; for example, a 12 FI 60, 12 FI 180 SW schedule consists of a block of 12 consecutive FI 60 s followed by a block of 12 consecutive FI 180 s in a repeating cycle. In Ludvig and Staddon, we found that pigeons will almost always show temporal tracking behavior on SW schedules, pausing longer following the longer intervals and shorter following the shorter intervals. The primary exceptions we found were schedules in which each interval was presented only a single time per cycle (i.e., single alternation). Based on work with more familiar types of discrimination (Hearst, 1962; Hunter, 1918, 1920; Livesey, 1969), animals might be expected to learn this sequential pattern and thus show the opposite of tracking, namely, a short pause following a long interval and vice versa.

These single-alternation schedules of reinforcement embody both sequential and temporal regularities. Innis (1981) found that pigeons do not generally show temporal tracking behavior on these single-alternation SW schedules. Ludvig and Staddon (2004) replicated this result with a 3:1 ratio between the long and short interval durations and also studied the development of this behavior across sessions. When exposed to these schedules, the pigeons started by tracking the prevailing interval for the first few days. After about nine sessions, pigeons stopped tracking the two intervals and did not show differential pausing for the remainder of training even after 35 sessions or more. Oddly, at no point in either experiment did pigeons perform what would seem to be the optimal behavior: pausing longer during the longer interval (i.e., following the short one) and vice versa.

Under other conditions, pigeons are able to learn an alternation sequence. A classic body of comparative research has explored the performance of many species (including rats, pigeons, cats, dogs, raccoons, rabbits, monkeys, and human children) on both the single- and the closely-related double-alternation problems (Hearst, 1962; Hunter, 1918, 1920, 1928; Khavari, 1970; Livesey, 1965, 1969; Travis-Neideffer, Neideffer, & Davis, 1982; B. A. Williams, 1971a, 1971b, 1976; F. E. Williams, White, & Messer, 2002). In these cases, a single-alternation task involved performing one of two responses (e.g., turning right or left in a maze) in an alternating fashion (RLRLRL). A double-alternation task involved performing each response twice in succession (RRLLRRLL). All species tested were able to master the single-alternation task, but there were significant species differences in the ability to master a double-alternation task. Pigeons show single-alternation learning for spatial location or color (Hearst, 1962; B. A. Williams, 1971a, 1971b), but are unable to learn a double-alternation sequence (B. A. Williams, 1976). Keller (1973) even found that pigeons can learn a single-alternation interval sequence when the two interval durations are 20 s and 180 s. “Correct” performance by pigeons on this sort of single-alternation task involves employing some type of a “win-shift, lose-stay” strategy (Randall & Zentall, 1997); animals must learn that the immediately preceding event or response will not be reinforced again.

A potential mechanism for learning any alternation pattern is for animals to learn a pair of sequential dependencies in the form of discriminative cues. That is, for an ABABAB repetitive sequence, animals can learn that response A cues response B and response B cues response A. In the case of a single-alternation interval schedule, such discriminative cuing would entail learning that long intervals cue short intervals and short intervals cue longer intervals. These discriminative stimuli would produce effects on pausing that are the exact opposite of temporal tracking and would result in longer pauses following shorter intervals (i.e., during longer intervals) and shorter pauses following longer intervals (i.e., during shorter intervals). As noted above, in previous work with single-alternation schedules (Innis, 1981; Ludvig & Staddon, 2004), this pattern of results did not emerge—pigeons paused similarly after both interval durations, showing neither tracking nor cuing.

Given that animals can learn a single-alternation pattern and that animals usually show temporal tracking with a lag of one interval, the nontracking behavior on single-alternation SW schedules may be best attributable to a two-process account whereby both temporal tracking and discriminative cuing are occurring with opposite effects. Temporal tracking produces the tendency to follow the interval and pause longer following longer intervals, whereas discriminative cuing produces the tendency for anticipatory pausing with longer pauses following the shorter interval. These two opposing tendencies would then cancel each other out, resulting in undifferentiated pause durations (i.e., nontracking behavior).

A competitive interaction between alternation learning and temporal tracking is not the only possible explanation for the pattern of responding seen on single-alternation interval schedules. An alternative mechanism to one-back temporal tracking is a long-term averaging “strategy”¹ according to which the pigeon's pausing is controlled by the mean (arithmetic or otherwise) of the previous few intervals. Wynne, Staddon, and Delius (1996) discuss several possible moving-average models for the relation between previous interfood intervals and the current PRP. For example, their simplest moving-average model predicts that the pause (P) on interval n + 1 should be a function of the unweighted mean of the previous M intervals (I):

where M is the number of previous intervals that play a role in controlling the pause, I is the duration of those intervals, and A is a constant less than 1. For any even M ≥ 2, in the single-alternation case, this model predicts identical pausing following both long and short intervals because the average of the previous M intervals never changes. In this framework, the degenerate case, where M  =  1, reduces to one-back tracking or simple linear waiting.

With a single-alternation sequence in place, one-back tracking is the worst possible strategy, resulting in longer pauses during the shorter intervals (thus potentially delaying reinforcers) and shorter pauses during the longer intervals (thus increasing the amount of responding emitted for a single reinforcer). As a result, animals may start with a default strategy of one-back tracking (M  =  1) in the initially unknown environment, but switch to an averaging strategy (with an even M ≥ 2) when confronted with an alternation schedule. In the latter instance, the pigeons would pause for an amount that is appropriate for an average of the two different intervals all the time. This idea is tantamount to the claim that when the default strategy (M  =  1) produces distinctly suboptimal results, pausing comes to be controlled by a larger window of past intervals.

If animals are indeed switching from temporal tracking to an averaging strategy on these single-alternation interval sequences, then the nontracking behavioral pattern displayed after the first few days should be impervious to changes in the interval durations. In the model offered above, a switch in interval durations would alter the absolute durations of the elicited pause, but would not change the size of the window of previous intervals controlling the pause. As a result, the actual duration of the pauses may change, but the nondifferential pausing pattern should be conserved regardless of the ratio or absolute durations of the two intervals. If, however, animals were really learning the alternation sequence and the nontracking behavioral pattern reflected a balance between this alternation learning and temporal tracking, then altering the interval durations should alter the balance between these two processes. At the outset of the new condition, animals could not have learned the new discriminative cues (for the new interval durations), but the tracking process would remain intact. For single-alternation conditions, we would expect renewed temporal tracking with each new pair of intervals that would gradually dissipate (and perhaps even reverse) as the new discriminative cues are learned. Thus, in this experiment, we manipulated both the ratio between the two intervals and the absolute interval durations in three single-alternation SW conditions.

A different set of predictions about the effects of changing the absolute and relative interval durations on these SW schedules arises from the notion of timescale invariance. The principle of timescale invariance claims that temporal dependent variables, such as PRP or peak time, should always show similar functions in timing experiments independent of the absolute timescale of the schedule contingencies (Gallistel & Gibbon, 2000; Gibbon, 1977, 1991). Examples abound in the literature on animal timing, including results from fixed intervals (Dews, 1970; Schneider, 1969), DRL schedules (Staddon, 1965), the peak procedure (Roberts, 1981), and the temporal bisection task (Church & Deluty, 1977). There have been suggestions, however, that this principle does not hold across very wide time scales nor for all measures of timing (Staddon & Higa, 1999; Zeiler & Powell, 1994). This principle makes a very straightforward prediction for behavior on a single-alternation SW interval schedule: If the ratio between the two intervals is maintained, manipulating absolute interval duration should have no effect on either acquisition or asymptotic performance.

The two main goals of this experiment were to further disentangle the roles of temporal tracking and alternation learning on SW schedules of reinforcement, and to test directly for timescale invariance. Pigeons were exposed to a series of single-alternation interval schedules with different durations for the two component intervals. Three pairs of interval durations (30 s and 180 s; 5 s and 180 s; 5 s and 30 s) were used as the short and long intervals in the single-alternation sequence. If pigeons shift from temporal tracking to a long-term averaging strategy with repeated exposure to a single-alternation schedule, then they should show nontracking behavior that is learned across sessions for all three pairs of intervals. In contrast, if pigeons' nontracking behavior on these schedules (Innis, 1981; Ludvig & Staddon, 2004) reflects a balance between alternation learning and temporal tracking, then we might expect pigeons to show anticipatory behavior when conditions are altered to facilitate alternation learning. We used a large (36:1) ratio between long and short interval durations and a very brief “short” interval duration (5 s) as two manipulations aimed at facilitating alternation learning on these schedules. Finally, if the principle of timescale invariance also holds for these dynamic timing schedules, then the absolute durations of these intervals should not affect the behavioral patterns displayed by the pigeons. So long as the relative durations (i.e., the ratio between the interval durations) are constant (in the 30/180 and 5/30 pairs), the behavior should be the same.

Method

Subjects

Six male pigeons (Columba livia) participated in this α experiment. Subjects were 2 to 5 years old and were kept at 85% of their ad libitum weight throughout the experiment. In their home cages, pigeons had unlimited access to water. All pigeons had previous experience with interval schedules of reinforcement. Subjects were treated in accordance with the ethical guidelines laid out by the American Psychological Association for research with nonhuman subjects.

Materials

Three operant chambers were built out of 24-gal (90.84 L) plastic storage containers (Rubbermaid® Roughneck) with a height of 31 cm and a plastic grid floor measuring 37 cm by 46 cm. A touchscreen-equipped computer monitor was located in front of a 20 cm by 27 cm hole in the front wall of the chamber. Reinforcers consisting of 2-s access to mixed grain (Purina® ProGrain for Pigeons) were delivered using a Coulbourn Instruments feeder (Allentown, PA, Model #E14-10) through a 5 cm by 4.5 cm hole on the right side of the container, 8 cm off the floor and 8.5 cm from the front of the chamber. A small light illuminated the inside of the food cup whenever a reinforcer was available. A fan mounted on the back wall of the chamber provided ventilation and masking noise.

Stimuli were presented on a 13 in. VGA Multisync A500+ monitor located approximately 1 cm behind a Carroll Touch Technology 13 in. infrared (IR) touch screen. Stimuli consisted of a white disk (radius  =  40 pixels—termed the key) on a black background that appeared on the monitor 18 cm off the ground with a diameter of 2.5 cm. Responses to a rectangle with sides measuring 120 pixels by 140 pixels around the key were recorded; all other responses were discarded. Custom software written in Borland® Delphi^TM 5.0 (Borland Software Corporation, Scotts Valley, CA) was run on three NEC© Pentium-II computers (one per chamber) running the Microsoft® Windows® 95 operating system to control presentation of stimuli and reinforcers as well as record responses. The temporal resolution of the full system was 50 ms, limited by the 20-Hz refresh rate of the touch screen.

Procedure

All sessions began with a 100-s start delay during which no stimuli were displayed on the screen followed by a “free” (response-independent) food presentation. Including this first food delivery, there were a total of 65 reinforcers available during each daily session. The remaining 64 reinforcers were all delivered according to an FI schedule whose criterion interval duration varied according to the SW schedule in place that day (see Table 1). The first four trials of each daily session were considered “warm-up” trials, and those data were discarded. On each trial, the first key peck by a pigeon that followed the expiration of the current criterion interval was reinforced by 2 s of access to the food hopper. The key light was extinguished during the period of reinforcement.

Table 1. Detailed description of each phase in the experiment. From left to right, the first four columns contain phase number, schedule administered, number of sessions, and pertinent result. The final four columns contain the mean pause (± SEM) for the first 15 sessions and last 5 sessions of each phase, split by the duration of the previous interval (long or short).

Phase	Reinforcement schedule	Total sessions	Result	Mean pause (s) ± SEM (First 15 sessions)		Mean pause (s) ± SEM (Last 5 sessions)
				After short FI	After long FI	After short FI	After long FI
1	Group 1: 12 FI 30, 12 FI 180	24	All: Tracking	5.7 ± 0.8	9.2 ± 1.6	7.1 ± 1.6	10.4 ± 0.9
	Group 2: 12 FI 5, 12 FI 180			2.7 ± 0.2	3.6 ± 0.1	2.5 ± 0.1	3.1 ± 0.8
2	Group 1: 1 FI 30, 1 FI 180	21	G1: No tracking	10.7 ± 1.9	11.8 ± 1.9	8.8 ± 0.9	8.7 ± 0.6
	Group 2: 1 FI 5, 1 FI 180		G2: Antitracking	4.8 ± 1.5	2.9 ± 0.5	8.4 ± 3.0	2.3 ± 0.5
3	All: 1 FI 5, 1 FI 30	30	All: Antitracking	3.3 ± 0.4	2.5 ± 0.3	7.1 ± 1.4	2.7 ± 0.4
				3.9 ± 0.8	2.7 ± 0.7	7.1 ± 2.4	2.3 ± 0.4
4	All: 1 FI 30, 1 FI 180	20	G1: Antitracking	7.3 ± 1.1	6.3 ± 0.8	8.5 ± 1.0	6.1 ± 0.9
			G2: No tracking	8.7 ± 1.4	8.9 ± 1.2	9.3 ± 1.7	9.5 ± 2.0

Table 1 details the exact schedules to which pigeons were exposed for all four phases that comprise the experiment. In the first phase, pigeons were split into two groups of 3 pigeons each. Both groups received a schedule of the form 12 FI Short, 12 FI 180, that is, 12 consecutive short FIs, followed by 12 consecutive FI 180 s in a repeating cycle. Sessions began at a random point in the cycle and continued until the pigeons had received all the reinforcers for that day. The first group (Pigeons P7257, P1853, and P7207) had 30 s as the short interval, resulting in a 12 FI 30, 12 FI 180 schedule. The second group (Pigeons P411, P7272, and P7227) had 5 s as the short interval, resulting in a 12 FI 5, 12 FI 180 schedule. For the second phase, the pigeons were maintained in two groups and received a single-alternation schedule of the form 1 FI Short, 1 FI 180 s. Both groups received the same short interval duration (5 s or 30 s) that they previously had encountered in the first experimental phase.

For the third and fourth phases, the two groups were collapsed and all 6 pigeons received the same schedule. To determine whether the ratio of intervals (6:1 or 36:1) or the absolute interval duration of the shorter interval (5 s or 30 s) was the critical variable for producing any between-group differences in Phase 2, all pigeons received a 1 FI 5, 1 FI 30 schedule in Phase 3. This schedule maintained the 6:1 interval ratio that one of the groups had received previously while combining this ratio with the 5-s short interval that the other group had been receiving. If the critical variable for any observed difference in Phase 2 was interval ratio, then the behavior of all subjects in this phase should be qualitatively similar to that of the 1 FI 30, 1 FI 180 group in Phase 2. If, however, the critical variable was the absolute duration of the shorter interval, then the behavior of all subjects in Phase 3 should be similar to that of the 1 FI 5, 1 FI 180 group from the previous phase. If the individual reinforcement histories were crucial, then behavior should be consistent within groups but different between groups.

In the final phase, all pigeons were placed on a 1 FI 30, 1 FI 180 to see whether any learning from Phase 3 might transfer to the longer intervals. In addition, this phase would determine if performance changes observed during Phase 3 were caused by the change in reinforcement schedule and did not simply result from greater exposure to single-alternation schedules. Over the course of the experiment, a pair of feeder malfunctions and touch screen breakdowns caused the loss of data for days 7 and 9 in Phase 2 for Pigeon P1853 and day 16 in Phase 4 for Pigeon P7272, and forced a 3-week hiatus that intervened between the 18th and 19th sessions of Phase 3 for all 6 pigeons.

Phases were terminated on the basis of two major criteria: (a) visual inspection of the daily data revealed relatively stable performance and (b) previous experience with experiments using similar schedules. Because our primary goal for this experiment was evaluating differences during the acquisition (not asymptotic) period following transitions in reinforcement schedule, a strict stability criterion was not relevant.

Results

The primary dependent measure in this experiment was postreinforcement pause (PRP)—the duration from the end of the reinforcer until the first key peck emitted by a pigeon. Other dependent measures, including the break point or start time (Cheng & Westwood, 1993; Church, Meck, & Gibbon, 1994; Schneider, 1969), overall response rate, and run rate, were considered, but these results were plagued with interpretive difficulties. The calculation of all three of these measures requires taking into account responding throughout the interval. In this experiment, with a mixture of intervals of different durations, the later portions of the longer intervals do not have direct counterparts during the shorter intervals (because those trials have ended and the reinforcers were already delivered). Differences in responding late in the longer intervals would be reflected in both the overall and run rates as well as the break points, even if responding during the early, directly comparable portions of the two intervals were identical. In addition, for the break point in particular, responding during the longer intervals did not resemble the putative two-state “break-and-run” pattern (Schneider, 1969) whose transition point the break point is supposed to estimate (see Figures 4 and 7). As a result of these considerations, these other measures of responding were not considered any further.

Figure 4. Mean responses emitted in 1-s bins across the final five sessions of Phase 2 (1 FI Short, 1 FI 180 schedule).

Pigeons in the left column received 30 s as the Short interval, whereas those in the right-hand column received 5 s as the Short interval.

Note the logarithmic x axis.

Figure 7. Mean responses emitted in 1-s bins across the final five sessions of Phase 3 (1 FI 5, 1 FI 30 schedule).

In the immediately preceding 1 FI Short, 1 FI 18-0 condition (Phase 2), pigeons in the left column received 30 s as the Short interval, whereas pigeons in the right column received 5 s as the Short interval.

Note the logarithmic x axis.

Figure 1 shows mean PRP calculated across all sessions in Phase 1 plotted as a function of trial into the SW schedule for each pigeon (see also the final columns of Table 1). The top row of graphs represents the overall mean pause for the pigeons in the two groups (dark line) as well as the actual criterion interval presented to the pigeons (dashed lines). Note the different y-axis values for the different groups of pigeons and the scheduled interval duration. The remaining three rows show data from individual pigeons. The 3 pigeons in the left column received the 12 FI 30, 12 FI 180 schedule, whereas the 3 pigeons in the right column received the 12 FI 5, 12 FI 180 schedule. As can be seen in Figure 1, 5 of the 6 pigeons show clear temporal tracking, pausing longer following the longer intervals and shorter following the shorter intervals. Pausing tended to increase across the first few longer (180-s) intervals following the transition from the shorter interval, before flattening out until the short intervals began again. This pattern held for pigeons in both groups, independent of the duration of the shorter interval. The 6th pigeon (P1853) showed little evidence of temporal control by the prevailing interval duration. Overall, the pauses were much shorter for the 3 pigeons that received 5 s as the short interval duration than for the pigeons that had 30 s as the short interval duration.

Figure 1. Postreinforcement pause (averaged across the 24 daily sessions) in Phase 1 plotted as a function of trial within the SW cycle.

The 3 pigeons in the left column received the 12 FI 30, 12 FI 180 schedule, whereas the 3 pigeons in the right column were exposed to the 12 FI 5, 12 FI 180 schedule.

In the top row, the dark, solid line depicts the mean pause for the 3 pigeons in that group, whereas the dashed line represents the programmed fixed interval in each trial.

Note that different y axes are used to display the programmed intervals and the postreinforcement pauses.

Figure 5. Postreinforcement pause following short or long intervals for the 1 FI 5, 1 FI 30 schedule (Phase 3), averaged across the final five sessions. With the immediately preceding 1 FI Short, 1 FI 180 schedule (Phase 2), the short FI was 30 s for the 3 pigeons on the left and 5 s for the 3 pigeons on the right.

Means are plotted + SEM, calculated as the standard error across the means for each of the final five sessions.

This pattern of results was also apparent during the first 15 sessions of this phase. The fifth and sixth columns of Table 1 present the group mean PRP following all 12 short or long intervals in each cycle. Pigeons showed temporal tracking even in those initial sessions, pausing longer following the longer intervals independent of the shorter interval duration. The tabular data are less clear than Figure 1 (especially for the 1 FI 5, 1 FI 180 group) because they present the average pause after all 12 longer intervals, thereby including the first few longer intervals after a transition in which there was not as pronounced an increase in pausing.

In the first single-alternation phase (Phase 2), all pigeons were presented with a schedule of the form 1 FI Short, 1 FI 180 (where the short interval was either 5 s or 30 s) for 21 sessions. The mean pauses averaged across the final five sessions in this phase for both groups as well as for individual pigeons are shown in Figure 2. The group on the left had 30 s serve as the short interval (for a resultant schedule of 1 FI 30, 1 FI 180), whereas the group on the right received 5 s as the short interval duration (for a resultant schedule of 1 FI 5, 1 FI 180). Over these final five sessions, all 3 pigeons that received the 30-s short interval showed virtually no effect of interval duration, pausing for an equivalent duration regardless of the length of the previous interval. A very different pattern emerged for the 3 pigeons that received 5 s as the short interval. As can be seen in the right panel of Figure 2, these pigeons paused longer following the shorter 5-s interval than following the longer 180-s interval. This “antitracking” pattern meant that the pigeons were pausing longer during the longer interval (i.e., after the shorter interval), indicating that they learned the single-alternation pattern. Pigeons P7272 and P7227 showed this effect to a much greater extent than Pigeon P411, but the effect was in the same direction for all the pigeons. The former 2 pigeons even showed pausing (12.8 s and 9.6 s) that was longer than the 5-s interval duration that they had experienced immediately prior. As we will see repeatedly below, Pigeon P411 seems least sensitive to the duration of current intervals and its responding does not exhibit temporal control in the same manner as the other 5 pigeons.

Figure 2. Postreinforcement pause following the short or long intervals on the 1 FI Short, 1 FI 180 schedule (Phase 2), averaged across the final five sessions. For the 3 pigeons plotted on the left, the short FI was 30 s, whereas for the 3 pigeons plotted on the right, the short FI was 5 s.

Means are plotted + SEM, calculated as the standard error across the means for each of the final five sessions.

Figure 3 shows how the mean PRP developed across sessions for both groups of pigeons with this 1 FI Short, 1 FI 180 schedule (Phase 2). For the first few days, all pigeons in both groups showed some temporal tracking behavior, pausing slightly longer following the longer 180-s interval and shorter following the shorter interval. For the pigeons receiving the 1 FI 30, 1 FI 180 schedule (left panel), the duration of their pauses converged after about 11 days and remained nondifferential for the remaining 10 days (nontracking). For the pigeons receiving the 1 FI 5, 1 FI 180 schedule (right panel), the duration of their pauses also converged, but more quickly, within the first four sessions. Starting with the seventh and eighth sessions, their pauses following short and long intervals diverged more sharply; however, they were now pausing longer following the short intervals and vice versa. Pigeons P7272 and P7207 showed the effect earlier and more clearly than Pigeon P411, but as the zoomed inset shows, even Pigeon P411 paused longer following the shorter interval over the final eight sessions. This antitracking behavioral pattern persisted and continued to grow considerably in magnitude throughout the remaining 14 days of training in this phase of the experiment.

Figure 3. Mean postreinforcement pause following short or long intervals for the 21 daily sessions for both groups of pigeons in Phase 2.

The first data point is the mean pause for the first session; all subsequent points are the mean pauses for two sessions.

All 6 pigeons received a schedule of the form: 1 FI Short, 1 FI 180. In the left column, the short interval was 30 s, and in the right column, the short interval was 5 s.

The zoomed inset for Pigeon P411 presents pausing over the final eight sessions with a rescaled y axis.

In addition to the PRP, we explored the postreinforcement distribution of all responses made by the pigeons during the interfood intervals. Figure 4 plots these distributions for the final five sessions of Phase 2. As can be seen in the left panel, all 3 pigeons that received 30 s as the short interval showed nondifferential behavior during the first 30 s of every trial, regardless of the duration of the previous interval. On the longer, 180-s intervals (i.e., after FI 30), all the pigeons showed an increasing rate of responding through the first 30 s; after the 30-s point, the 3 pigeons showed slightly different patterns. Pigeon P7257 maintained a high rate of responding through the remainder of the interval; the rate of responding of Pigeon P1853 tapered off a little (until about 50 s into the interval) and then remained approximately constant; and the responding of Pigeon P7207 showed a considerable drop-off after the 30-s point, before gradually increasing for the rest of the interval.

The 3 pigeons that had 5 s as the short interval duration (right panel) displayed a very different pattern of behavior. They responded at a considerably higher rate in the first few seconds of the short, 5-s interval than the longer, 180-s interval—a pattern again most visible in the responding of Pigeons P7272 and P7227, but still apparent for Pigeon P411. During the remainder of the 180-s interval, the responding of these 3 pigeons showed a drastic drop in rate following the initial burst of responding, before gradually increasing from the 15-s mark until the end of the interval. These results accord with the pausing results (see Figures 2 and 3 and Table 1) in that pigeons in the 30-s group show nondifferential responding (and similar pausing) early in the interval, whereas pigeons in the 5-s group show much higher responding (and shorter pauses) in the early part of the 5-s intervals that follow the 180-s intervals. Thus pigeons seem to correctly “anticipate” the upcoming interval when the short interval is 5 s, but not when the short interval is 30 s after 21 sessions with this 1 FI Short, 1 FI 180 schedule.

To determine whether the large ratio (36:1) or the very short “Short” interval duration (5 s) was responsible for the difference between the two groups in Phase 2, all pigeons were then exposed to a 1 FI 5, 1 FI 30 schedule (Phase 3). This schedule simultaneously presented pigeons with both the smaller 6:1 ratio and the short interval duration of 5 s. Figure 5 plots the mean PRP, averaged across the final 5 out of 30 sessions for this schedule. This figure clearly illustrates how all 6 pigeons learned the alternation pattern with the 1 FI 5, 1 FI 30 schedule, showing longer pauses following the shorter, 5-s interval duration (i.e., during the longer interval), and shorter pauses following the longer, 30-s interval. Pigeon P411 once again shows an effect in the same direction as the other 5 pigeons, but to a much smaller extent. The pigeons are split into two columns according to the reinforcement schedule in the previous phases (Phases 1 and 2: Short was 30 s or 5 s). These results held for both groups of pigeons and were qualitatively similar to the results from the 1 FI 5, 1 FI 180 group in the immediately preceding Phase 2, indicating the absolute interval duration of the short interval (i.e., 5 s) was likely responsible for the differences observed earlier in Phase 2 (see Figure 2).

As was the case in the earlier phase, this antitracking or anticipatory behavior developed across sessions. Figure 6 shows the pause after FI 5 or FI 30 for the two groups of pigeons (plotted separately according to the schedule received in the previous phase) for all 30 sessions in this phase. The gap between Days 18 and 19 (session Blocks 9 and 11) represents a period of 3 weeks during which no experimental work was conducted. For the first few days, all 6 pigeons showed little to no differential effect of the previous interval duration. They paused for an approximately equivalent amount whether the last interval was 5 s or 30 s. After the first few days, the pauses after the short FI 5 s were considerably longer than the pauses following the longer FI 30 s. The magnitude of this difference continued to grow until the experiment was interrupted after Day 18. After this break, the antitracking effect was still present but smaller in magnitude; this difference grew again across the remaining 12 days that pigeons received this schedule. Of particular note, even the 3 pigeons that showed the antitracking pattern in Phase 2 and thus displayed clear evidence for learning the alternation pattern (Short was 5 s group, right column) started with a minimal difference in their pauses.

Figure 6. Mean postreinforcement pause following short or long intervals for the 30 daily sessions in two-session blocks for both groups of pigeons in Phase 3.

All pigeons received a 1 FI 5, 1 FI 30 schedule of reinforcement.

For the pigeons in the left panel, the previous schedule they received was 1 FI 30, 1 FI 180 (in Phase 2), and for the pigeons in the right panel, the previous schedule was 1 FI 5, 1 FI 180.

The zoomed inset for Pigeon P411 presents pausing over the final eight sessions with a rescaled y axis.

Examination of all responses that the pigeons made during the interfood interval paints a very similar picture to the PRP data. In Figure 7, the distribution of all responses made by the individual pigeons is plotted as a function of time since the previous reinforcer. The mean data from both groups (again split by schedule history) are similar and, with the exception of Pigeon P411, quite representative of the individual pigeon data. On the longer, 30-s intervals (i.e., after FI 5 s), pigeons show a much lower rate of responding early in the trial than during the shorter, 5-s intervals (i.e., after FI 30 s). Five out of the 6 pigeons show this effect, whereas Pigeon P411 retains its anomalous status, failing to show differential responding in the first few seconds following reinforcement. Later in the trial, responding tapers off quite sharply, leaving a peak of responding 4 to 5 s into the interval, before recovering gradually from the 10-s point until reinforcement is received around the 30-s mark. Thus pigeons learned the alternation pattern and correctly anticipated (i.e., by responding more vigorously in the early part of the interval) the short interval that followed the long interval.

When the pigeons were then exposed (or reexposed) to a 1 FI 30, 1 FI 180 schedule (Phase 4), each pigeon's individual reinforcement history played a role in determining the pattern of responding exhibited. Figure 8 depicts the mean PRP following the short (30-s) and long (180-s) intervals, averaged over the final five sessions for this phase. In the left panel, the 3 pigeons that previously received this exact 1 FI 30, 1 FI 180 schedule (in Phase 2) showed an antitracking behavioral pattern, pausing longer following the shorter, 30-s interval. In contrast, in the right panel of Figure 8, the pigeons that were naive to this pair of interval durations paused for an equivalent amount regardless of the preceding interval.

Figure 8. Postreinforcement pause following short or long intervals for the 1 FI 30, 1 FI 180 condition (Phase 4), averaged across the final five sessions.

In the earlier 1 FI Short, 1 FI 180 condition (Phase 2), pigeons in the left column received 30 s as the short interval, whereas pigeons in the right column received 5 s as the short interval.

Means are plotted + SEM, calculated as the standard error across the means for each of the final five sessions.

The development across sessions of pausing behavior on this presentation of the 1 FI 30, 1 FI 180 schedule (Phase 4) is displayed in Figure 9. The 3 pigeons that had previously received this schedule (left panel) initially show very little difference in pausing. This nontracking behavior corresponds with the pattern they exhibited at the end of their previous exposure to this phase (see Figures 2 and 3), but differs from the pattern they showed at the end of the immediately preceding 1 FI 5, 1 FI 30 condition where they showed anticipatory responding (see Figures 5 and 6). After the first six to eight sessions, though, these pigeons begin to display anticipatory pausing, waiting longer to initiate responding following the shorter interval. The 3 pigeons that were naive to this schedule provide a sharp contrast to these results (right panel). These pigeons also start by showing no differential responding following the different intervals, but they continue to show little to no difference through 20 sessions of testing. Both of these groups of pigeons had responded identically with the prior 1 FI 5, 1 FI 30 schedule. Thus it seems that with sufficient and appropriate training, pigeons can and do learn to show anticipatory pausing, even with the 1 FI 30, 1 FI 180 schedule. Such anticipatory behavior, however, when it does occur, takes considerably longer to develop than with shorter interval values (e.g., Phase 3: 1 FI 5, 1 FI 30).

Figure 9. Mean postreinforcement pause following short or long intervals for the 20 daily sessions for both groups of pigeons in Phase 4, averaged in blocks of two sessions.

All pigeons received a 1 FI 30, 1 FI 180 schedule of reinforcement.

For the pigeons in the left panel, the short interval during the earlier 1 FI Short, 1 FI 180 condition (Phase 2) was 30 s, meaning that they had previously received 21 sessions with the exact 1 FI 30, 1 FI 180 schedule administered in this phase.

For the pigeons in the right panel, the previous short interval was 5 s.

Figure 10 summarizes the results from the three single-alternation phases in this experiment (Phases 2 through 4) by showing the development of the mean PRP across training in blocks of five sessions. Only the final 20 sessions (out of 21), divided into four blocks of five sessions, are included from the 1 FI Short, 1 FI 180 schedule (Phase 2). The top row presents data from the group of pigeons that had 30 s as the short interval in Phases 1 and 2, whereas the bottom row presents data from the group of pigeons that had 5 s as the short interval in the first two phases. We would like to highlight two key results that are evident in this figure. First, the 3 pigeons that were exposed to the 1 FI 30, 1 FI 180 schedule in Phase 2 (Group 1) did not show differential pausing on that first exposure. These pigeons eventually showed anticipatory pausing, but only after 25 to 30 sessions total exposure to the 1 FI 30, 1 FI 180 schedule (summed across Phases 2 and 4, with additional intervening experience with single-alternation schedules). As can be seen in Table 1, however, in the first 15 sessions of exposure to the 1 FI 5, 1 FI 30, this group of pigeons already exhibited substantial differences in their pause following the two intervals. In this instance, it seems that the long-run asymptotic behavior was similar in the different single-alternation phases for this trio of pigeons, but the short-run acquisition behavior (first 15 to 20 sessions with a given schedule) proved remarkably different. The speeded acquisition with the repeat of the 1 FI 30, 1 FI 180 schedule (Phase 4) could be equally well attributed to continued exposure to the same schedule from Phase 2 or the intervening 1 FI 5, 1 FI 30 schedule (Phase 3). In either case, however, the initial acquisition behavior on these two single-alternation SW schedules was undoubtedly influenced by the absolute interval duration (5/30 vs. 30/180).

Figure 10. Mean postreinforcement pause for both groups in Phases 2 to 4, averaged across blocks of five sessions.

Second, the 3 pigeons from Group 2 showed anticipatory behavior during the 1 FI 5, 1 FI 30 schedule (Phase 3), starting from the second block of five sessions (bottom-middle panel). In the subsequent, fourth phase (bottom-right panel), those very same 3 pigeons failed to show any differential responding following the two different intervals—even after 20 sessions of training. Here is a more striking example of how, despite already showing evidence for single-alternation learning, a change in absolute interval duration (and no change in the ratio of intervals) can abolish this pattern for at least 20 sessions.

Discussion

Pigeons can show single-alternation learning for a strictly alternating interval sequence. When the two intervals were sufficiently distinct from one another, the shorter interval was able to cue the presence of the longer interval (and vice versa), and pigeons showed alternation-learning behavior (see below for further discussion of experimental variables that may affect the extent to which intervals are “distinct”). The critical variable for this alternation learning was the absolute duration of the shorter of the two durations. When the shorter interval was 30 s, pigeons initially showed a learned, nontracking behavioral pattern for a total of over 25 sessions (combined over Phases 2 and 4 for Group 1). In contrast, when the shorter interval was 5 s (Phases 2 and 3), antitracking (i.e., anticipatory pausing) occurred in which pause durations were longer following short intervals and vice versa. These results suggest a plausible mechanism for behavior on single-alternation SW schedules and challenge the scope of timed behaviors to which the principle of timescale invariance extends.

Two explanations for the behavior of pigeons on these single-alternation SW schedules were suggested in the introduction: a transition from one-back temporal tracking to long-term averaging, and a balance between tracking and alternation learning. Our findings do not support the former explanation. In the moving-average model discussed earlier, the transition between tracking and averaging consists of a change in the size of the window of past intervals that controls pausing (from M  =  1 to M ≥ 2). If pigeons switched between these two modes of responding, they would start by showing one-back temporal tracking (M  =  1) before switching to the averaging strategy (M ≥ 2) and showing nondifferential pausing behavior on all conditions. At no point should the pigeons show “cuing effects” by pausing longer after the shorter intervals and vice versa. In most earlier research (Innis, 1981; Ludvig & Staddon, 2004) and the first time pigeons received the 1 FI 30, 1 FI 180 schedule (see Figure 10), pigeons initially show one-back temporal tracking before displaying a learned, nontracking behavior—a pattern that is compatible with this mode transition idea. For the two phases where the shorter interval was 5 s as well as the second iteration of the 1 FI 30, 1 FI 180 schedule for pigeons in Group 1, however, pigeons show anticipatory, antitracking behavior. This behavior is incompatible with both one-back temporal tracking (M  =  1) and with any long-term moving average (M ≥ 2).

Perhaps a more sophisticated averaging model would capture the anticipatory pausing results. As Killeen (1994) discusses and Wynne et al. (1996) acknowledge, the unweighted moving average of M intervals imposes an arbitrary and sharp edge to a biological memory process that is highly improbable. Both papers suggest that a more likely model for memory would weight more recent intervals (or responses) more strongly and older intervals less strongly. One possible weighting function would have the pause (P) on trial n + 1 determined by an exponentially weighted moving average of previous intervals:

where A and γ are constants less than 1 (Wynne et al., 1996). This equation effectively captures the dynamics of pausing when the PRP is largely, but not entirely, determined by the duration of the previous interval. In Phase 1, we see evidence for this dynamic in the 12 FI Short, 12 FI 180 schedules where pausing increases across successive presentations of the longer interval (cf. Figure 1). Killeen also uses this function as the basis for determining the coupling coefficients in his theory on the mathematical principles of reinforcement (Killeen & Sitomer, 2003). For any given sequence, the largest weight is assigned (and greatest effect on pausing attributable) to the most recent interval (or response) with earlier intervals receiving exponentially decreasing weights. This rank ordering of the largest weight to the most recent interval is a feature that is conserved regardless of the weighting function. Consequently, one prediction that all decaying average models share is that under no circumstances should anticipatory pausing be possible. Without an additional process (like cuing), no decaying weighted-average model can predict the anticipatory results in which a longer pause followed a shorter interval and/or vice versa.

The idea that behavior on single-alternation interval schedules reflects a balance between alternation learning and temporal tracking better describes the current dataset. At the beginning of exposure to any schedule, before pigeons have had an opportunity to learn the alternation sequence, one-back temporal tracking dominates and pigeons pause longer following the longer intervals and shorter following the shorter intervals (see Figures 3 and 6). After a few days of exposure to the single-alternation schedules, pigeons begin to learn that short intervals serve as discriminative cues for longer ones and vice versa. With most of the pairs of intervals tested thus far (20/60, 30/180, 30/90, 60/180), these two processes fall into balance and pigeons show nontracking behavior after the first few days (see Innis, 1981; Ludvig & Staddon, 2004). When the two intervals are sufficiently distinct from one another, however, this discriminative cuing dominates, and pigeons display the antitracking, anticipatory behavioral pattern.

This potential explanation raises the question of what exact parameters for the interval durations constitute “sufficiently distinct,” such that pigeons will show cuing effects on these schedules of reinforcement. Two main variables that play a role in determining this criterion are the short interval duration (I) and the ratio between the two intervals (k). In this experiment, we found that I  =  5 s is sufficient for cuing to dominate and for pigeons to show alternation learning on these single-alternation SW schedules. When k  =  3 (Innis, 1981; Ludvig & Staddon, 2004) or 6 (as in the present experiment), nontracking behavior is usually observed (or a balance between cuing and tracking), except when I  =  5 s. When k  =  36, cuing dominates and pigeons show anticipatory, antitracking behavior (although I  =  5 s in all conditions where k  =  36). To fully characterize the effects of absolute interval duration (I) and ratio (k) would require a more complete parametric manipulation than has been conducted thus far.

An alternate explanation for what pigeons are learning in these single-alternation situations is that they form a “chunk” whenever two reinforcements come in quick succession. That is, pigeons learn to pause longer if multiple reinforcements have occurred in the past few seconds. Perhaps the first reinforcement enhances an inhibitory effect on responding of the second, later reinforcement or, alternatively, pairs of successive reinforcements can serve as a distinct perceptual unit for the control of pausing behavior. In either case, according to this hypothesis, the only important variable for determining “alternation learning” is the duration of the shorter interval duration. Albeit similar to the “cuing” explanation above, the key difference is that this “chunking” account does not require the sequential dependency between shorter and longer intervals—whenever any two reinforcements are close enough together in time, responding is delayed and a longer pause ensues. This notion fits well with the data from most of the single-alternation conditions, but runs into difficulty with the data from Phase 1 of this experiment. When pigeons receive a 12 FI 5, 12 FI 180 schedule, there are multiple quick intervals in succession, yet pigeons do not show any evidence for increased pausing following pairs of these short interfood intervals (see Figure 1).

An interesting finding in Phases 3 and 4 (see Figure 10) is the apparent total lack of transfer of the alternation learning from one single-alternation interval sequence to another. Whenever pigeons were faced with a new pair of intervals in these SW schedules, they always started with a one-back temporal tracking strategy. This result is most remarkable at the beginning of Phase 3 (see also Figure 6) where the pigeons that displayed antitracking behavior at the end of Phase 2 (Group 2: Short  =  5 s) behaved no differently than those pigeons that displayed nontracking behavior throughout Phase 2 (Group 1: Short  =  30 s). These results indicate that either the pigeons showed very little transfer from one condition to the next, or that both groups of pigeons had equivalent alternation learning in the former condition (Phase 2), despite its differential expression in the pigeons' behavior. This observation leads us to suggest that the abstract term alternation learning might not be the best way of describing the preceding data. The discriminative cuing mechanism outlined earlier, whereby pigeons learn that this particular short interfood interval is followed by this particular long interfood interval, better jibes with the renewed tracking with each new pair of intervals, and could certainly suffice to produce this alternation learning-like behavior.

Finally, these results represent a direct violation of the principle of timescale invariance propounded in the realm of interval timing (Gallistel & Gibbon, 2000; Gibbon, 1977, 1991). This principle states that the properties of timing (speed of learning, error distributions) depend only on the relative durations of time intervals. These claims embrace both asymptotic timed behaviors and the acquisition of associative learning. In the present experiments, the absolute timescale of the interval durations (I) quantitatively and qualitatively changed the pattern of learning and timing displayed by the pigeons. When the ratio between short and long interval durations was held at 6:1 (k  =  6), pigeons showed alternation learning considerably more rapidly when I  =  5 s (1 FI 5, 1 FI 30) than when I  =  30 s (1 FI 30, 1 FI 180). One group of pigeons never showed alternation learning at all under the latter conditions, even after 20 sessions of training with I  =  30 s (Phase 4) and over 70 sessions of single-alternation experience altogether. No transformation (linear or otherwise) would allow the results from Phases 2 and 3 with Group 1, and Phases 3 and 4 with Group 2 to superimpose. Thus the principle of timescale invariance is not general.