Associative learning and timing

Abstract

Associative learning and timing are clearly inter-related, but are they separate processes or is timing a core part of the associative structure? Emerging research suggests that temporal information is acquired rapidly and that CR's are timed correctly from the start of associative learning. Moreover, specific temporal knowledge can be disclosed even in cases where CR's were not emitted. Timing is not only critical for CR timing, but also contributes to CR expression through the comparison of reinforcer rates, and through the formation of temporal maps. A conceptual framework is proposed in which timing is a core part of the content of associative learning.

Introduction

The idea that time is involved in associative learning has been an intrinsic part of thinking since temporal contiguity was suggested as essential for association formation by Aristotle. This idea was developed by the British associationist philosophers of the 19^th century and became foundational for modern experimental psychology. The idea that learning depends on temporal contiguity is often accepted as axiomatic [1-3]. In this view, time is important for the establishment of associations but is not part of what is encoded in the association. More recently, there has been considerable work showing that times are encoded in associative learning paradigms and determine the type, vigor and temporal patterns of behavior. We review this work and discuss how these findings open new theoretical possibilities for understanding the nature of learning.

Time affects acquisition speed and asymptote

There has been considerable research showing that temporal factors alter the speed of conditioned response (CR) emergence as well as the asymptotic level of responding. In general, the greater the temporal contiguity between the conditioned stimulus (CS) and unconditioned stimulus (US) the greater the conditioning, provided that CS onset occurs prior to US occurrence. This pattern is observed in autoshaping [4,5], goal tracking [6-9], eyeblink conditioning [10-12], conditioned suppression [13], salivary conditioning [14], paw flexion [15], and heart rate conditioning [16] paradigms. However, if CSs are too brief, conditioning is diminished, perhaps because the CS is not attended to on some trials, or that a very brief cue is not sufficient to recruit the specific CR [17].

In addition to affecting the likelihood or strength of CR's, the CS-US interval also affects the form of CR's. Holland [18] found that when a short duration auditory CS was paired with food, rats startled and jerked their heads to the CS. However, when the CSs were longer the dominant response was approach to the magazine. Thus, the probability of a specific response cannot be unambiguously taken as an index of the strength of learning, as different responses may be predominantly expressed to different duration CSs. Similar observations have been made about the impact of temporal variables on the form of CR's in fear conditioning [19], eyeblink conditioning [20], sexual conditioning [21], and other appetitive conditioning paradigms [22,23]. Thus, one should not conclude that changing contiguity of the CS and US necessarily changes the underlying learning.

Another temporal interval that affects conditioning is the duration of the intertrial interval (ITI). In appetitive conditioning in pigeons [4,5] or rodents [6-9] and in fear conditioning [13,24], longer ITIs result in stronger CR's than shorter ones. The ITI, though, does not seem to be the determining factor. When the time between trials (CS-US pairings) is held constant but additional USs are presented during the intertrial interval, conditioning is weakened [25] suggesting that the key variable is the US-US interval rather than the ITI. In standard conditioning protocols longer ITIs are associated with longer US-US intervals, and the latter interval seems to be the key feature determining the effect of trial spacing [26,27]. It is worth noting, however, that even in cases where the average rate of reinforcement is the same, or even is lower during the CS than during the ITI, anticipatory timing is still observed [28,29]. Thus, timing is apparent even in situations that are not advantageous for CR expression.

That the CS-US and US-US interval durations affect conditioning seems indisputable. It has been further claimed that the processes that underlie the learning and expression of CR's are determined by the ratio of these two intervals [27,30,31]. In other words, conditioning is determined by the degree to which the CS signals a reduction in the wait between USs. Such a view implies that these intervals are encoded and serve as the basis for the emergence of anticipatory CR's. The absolute duration of these intervals may also contribute to conditioning [6-8], providing yet another route for the CS-US and US-US intervals to impact on anticipatory CR's.

Learning time versus temporal modulation of conditioning

As described above there is no controversy over whether time affects conditioning. The controversy arises over how to interpret these effects. Does time modulate conditioning processes because contiguity affects the formation of associative bonds (that include temporal information) or is time the foundational content of the learning that determines the behavioral output [30-33]?

From the time of Pavlov it was known that CR's often coincide with the expected time of US presentation when a predictable CS-US interval is delivered. In eyeblink conditioning, conditioned lid closure slightly precedes the expected time of a shock or airpuff [e.g., 10,20]. In appetitive conditioning, mice and rats become more and more likely to put their head into a feeder as the expected time of reward approaches [6-9,23,34]. Likewise, when animals are trained to fear cues, behavioral (e.g., freezing, startle potentiation) and physiological (e.g., respiration) responses peak at the time that a particular cue signals the arrival of shock [35,36]. Further evidence that time is encoded during the initial learning comes from studies that look at CR timing when CR's first emerge. While good timing is not universally observed at the start of conditioning [34] it is often the case [7,37,38] that when CR's first emerge they are timed to anticipate the US. It has even been shown that times are encoded accurately before the first CR's emerge [39]. In addition, the fact that CR's are not well timed does not mean that times have not been encoded. Diaz-Mataix et al. [40] found that after fear conditioning with a CS-US interval freezing responses were poorly timed. However, a single reminder trial with a different CS-US interval triggered a reconsolidation process. If the CS-US interval remained the same on the reminder trial, reconsolidation was not triggered. This indicates that the reconsolidation was triggered by the mismatch between the remembered CS-US interval and the new one, indicating that the CS-US interval was encoded.

Another way to examine temporal expectations when CR's are not temporally graded is to use a transfer test and see whether behavioral effects in the transfer test are maximal at the previously trained times of US presentation. In a blocking procedure, for example, a CS1 is paired with a US during the first phase of training and then a novel CS2 is compounded with CS1 and pairings continue. Typically, the learned expectancy of the US during the first phase interferes with the CS2's capacity to evoke CR's. It has further been found in blocking procedures that changing the time at which the US is presented during the second phase attenuates the blocking effect [41,42], although this effect is not always obtained [43-45].

Similar results have been reported for transfer tests of overshadowing [two cues are presented simultaneously from the start of training; 46,47] and conditioned inhibition [conditioned inhibition is maximal at the time at which the US had been expected but was omitted during inhibitory training; 48-51]. Thus, it appears the CS-US temporal relationship is an important element of the learning that occurs in cue competition paradigms.

A related piece of evidence that times are encoded comes from demonstrations that animals can integrate information about time across separate experiences [52-54]. In second-order conditioning, a CS1 is followed by the US, and then CS1 is later paired with a novel CS2 in the absence of the US (Figure 1A). CS2 is now able to evoke CR's, even though it was never directly paired with the US. Associative theory proposes that CS2 inherits associative strength through mediated associations. Miller and colleagues provided an alternative account, the temporal coding hypothesis [55,56], which proposes that a temporal map is formed and connected together across the two learning episodes, similar to processes that occur with spatial maps. The connector is CS1, which is common between the two stages of training. The temporal map would encode the temporal relationships between the CS1, CS2, and US, as shown in Map A, in which the expected time of the US is simultaneous with CS2 onset. This leads to the prediction that second-order conditioning should be comparatively weak given that simultaneous conditioning generally results in weak anticipatory CR's (see above). Indeed, Cole et al. [57] confirmed that this is the case [see also 32,54 for related evidence in appetitive conditioning] by comparing standard second-order conditioning with trace second-order conditioning (Figure 1B). Trace conditioning resulted in weaker initial conditioning, and thus should support weaker mediated association transfer to CS2. Instead, CS2 resulted in more robust CR's in the trace group, consistent with temporal map formation. In Map B, CS2 occurs in a stronger predictive arrangement with the US in comparison to Map A. Evidence for temporal map formation has been found in a range of conditioning paradigms including overshadowing, blocking, and conditioned inhibition paradigms [e.g., 41,46,48,57,58,59]. In addition, it appears that temporal maps may be flexible in their scaling similar to spatial maps [60] and contain bidirectional representations [55].

Figure 1.

The formation of temporal maps in second-order conditioning. A. A standard second-order conditioning task in which first-order conditioning is delivered in a delay conditioning arrangement with CS1 followed directly by the US. In phase 2, the original CS1 is now followed by CS2, in the absence of any US presentations. The resulting Temporal Map A contains a layout of the CSs and the US in terms of their timing and order of occurrence. B. A trace-second-order conditioning task and the resulting temporal map. In trace conditioning, the CS1 and US in Phase 1 are separated by a gap.

Conclusions

All associative learning procedures involve CS and US events that unfold in time, and thus can be specified by the events (CSs and USs), their coincidence (pairing) and order of occurrence, the timing of events, and the probability of occurrence of each event. The key question raised by the evidence reviewed above is how to understand the relationship between timing and associative learning. Timing models such as scalar timing theory [61,62] account for specific interval learning, whereas rate expectancy theory [27,30] and information theoretic models [31,32,63-65] account for interval comparisons, and the temporal encoding hypothesis [55,56] for temporal maps. Each of the theories does a reasonable job of accounting for effects within its domain, albeit with some weaknesses. However, there are many gaps in the current state of the knowledge in the field, and many of the gaps occur at the interface of the different processes [66]. For example, are temporal maps formed through a separate process from associative learning? And, if so, are they formed in parallel or in series with associative processing? We describe two very different conceptualizations of these questions, shown in Figure 2.

Figure 2.

A conceptualization of the timing processes in associative learning. The pairing of CS and US in a contiguous forward relationship results in two potential processes. An associative learning process involves the learning of specific associations and temporal intervals that make up the conditioning paradigm. The combination of information informs CR expression and timing, providing the appearance of a temporal map. The second potential process involves the rapid formation of a rich temporal map that encodes the nature (location, modality, and intensity) of events, their durations, and their timing of occurrence. CR's are expressed when sufficient evidence suggests that effort should be allocated to responding.

One possibility is that information about time (coincidence, order, and timing of events) is an integral part of content of associative learning [e.g., 56]. In the associative view, very early in training (perhaps even during/after the first trial), CS-US presentations result in coincidence detection and determination of the order of events. This process is an essential determinant for conditioning as the contiguity and order of events has a major impact on learning. A second critical process is temporal processing, wherein the specific intervals between events is learned. This involves learning the CS-US and US-US intervals, both individually for guiding CR timing, and also their comparison for guiding CR expression. These specific associations contain the knowledge that makes it appear as though there is a temporal map. Future research to assess these processes should target these systems to determine their inter-relationship. And, future theories need to deal more explicitly with the interface of these processes.

Alternatively, temporal maps may be rapidly apprehended in the same way that spatial, auditory and visual properties of objects are apprehended and encoded as they are dynamically encountered. Specific information about events (location, modality, intensity, duration, and timing of events) can be extracted from those integrated map-like representations of the world. In this view, representations are not built up from the association of elements, rather information about elements is extracted from holistic representations of experiences that are anchored in space and time. Thus, the slow emergence of CR's is not because of a slow buildup of underlying learning, but because the reliability of the information needs to be assessed before action is guided by that information. Future research guided by this view might well focus on how representations of the stream of events changes over the course of exposure to conditioning protocols and how these changing representations translate into specific responses.

The study of timing and associative learning deals with fundamental principles of learning which broadly impact on human and animal behavior. We have learned much about these principles, but it is also clear that we have much yet to learn. Further examinations of the different timing processes in conditioning, both individually and in concert is critical for future research in the field. The evidence reviewed here suggests that rich temporal maps may be formed well before CR's are expressed, indicating the need for more sensitive measurements to disclose the nature of information encoding within associative learning paradigms.

Highlights.

• Temporal information is acquired rapidly in associative learning

• CR's are timed correctly from the start of associative learning

• Specific temporal knowledge can be disclosed even in when CR's are not emitted

• Rich temporal maps are formed during associative learning