Timing: An Attribute of Associative Learning

Abstract

The evidence reviewed in this paper suggests that when two events occur in spatiotemporal proximity to one another, an association between the two events is formed which encodes the timing of the events in relation to one another (including duration, order, and interval). The primary evidence supporting the view that temporal relationships are encoded is that subsequent presentation of one event ordinarily elicits behavior indicative of an expectation of the other event at a specific time. Thus, temporal relationships appear to be one of several attributes encoded at acquisition.

1. Introduction

The simplest form of associative learning is Pavlovian (i.e., classical) conditioning. In Pavlovian conditioning, an association is formed between an initially neutral stimulus and an unconditioned stimulus (US) such that subsequent presentation of the now-conditioned stimulus (CS) presumably activates an anticipatory representation of the US. Anticipation of the US causes the animal to emit a conditioned response (CR) appropriate for the specific US.

Pavlov (1927) and many subsequent researchers have identified numerous behavioral phenomena that arise within Pavlovian conditioning. Here we cannot convey the richness of the empirical corpus, but we will review briefly some of those key phenomena such as acquisition, cue competition (e.g., overshadowing), conditioned inhibition, and associative interference (e.g., retroactive interference) in which temporal relationships appear to be critical. The first goal of this review is to highlight the fact that temporal relationships are encoded as a part of any association by showing that they strongly influence these key Pavlovian behavioral phenomena. The behavioral phenomena to be described will be analyzed within the framework of the temporal coding hypothesis (TCH, e.g., Savastano & Miller, 1998), which is a set of hypotheses concerning how temporal information is used within any associative model. The tenets of the TCH can be summarized as follows: (a) close contiguity between events is both necessary and sufficient for the formation of an association; (b) the temporal relationship between the associated events is encoded as part of the association (also see Honig, 1981); (c) this temporal information plays a critical role in determining the topology, magnitude, and timing of the response elicited when one of the associates is subsequently presented; and (d) subjects can superimpose temporal relationships when they share a common element, even when the relationships were independently acquired, thereby allowing for the expression of temporal relationships between cues that were never actually paired together (i.e., temporal integration). The second goal of this review is to present recent refinements of the TCH concerning (a) the time at which temporal integration occurs, (b) the associative structure of temporal integration, and (c) the directional nature of the temporal coding.

2. Acquisition with different temporal arrangements

Pavlov (1927) was the first to investigate the effects of the CS-US interval on CR. He used four different temporal arrangements between the CS and the US. In forward delay conditioning, the CS is presented before the US and stays on until the US is presented. In forward trace conditioning, the CS is presented, and then the US occurs at some time after the termination of the CS. In simultaneous conditioning, the CS and US are presented at the same time. In backward [trace] conditioning, the US occurs such that it terminates prior to the onset of the CS. Pavlov found that simultaneous and backward conditioning do not produce appreciable CRs; that with trace conditioning, longer intervals between the CS and US produce weaker CRs; and that in forward delay conditioning the CR grows weaker as the interval from CS onset to US onset is increased. These results have been repeatedly confirmed in subsequent experiments (e.g., Cooper, 1991; Mackintosh, 1983; Matzel, Held, & Miller, 1988; Rescorla, 1988).

The contiguity principle, to which Pavlov (1927) subscribed, assumes that good contiguity is merely a catalyst for forming associations, and differences in responding to CSs trained with different temporal arrangements are the result of differences in the strength of the resultant associations. The closer the two stimuli are in time, the greater the strength of the association will be, and consequently the more robust the CR to that CS. It is noteworthy that this principle seems well suited for most temporal arrangements in simple excitatory Pavlovian conditioning. However, it is challenged by the weak conditioned responding observed following simultaneous conditioning, which according to the contiguity principle should yield maximal responding because simultaneity is synonymous with perfect contiguity (i.e., the CS and US could not be closer in time). Similarly, according to the contiguity principle, backward trace and forward trace conditioning with identical intervals between the CS and US should produce equally strong associations and hence result in equally strong conditioned responding. However, forward conditioning consistently results in more robust responding than backward conditioning. Thus, the contiguity principle provides an incomplete account regarding the role of contiguity in conditioning.

The TCH better accounts for the behavior observed with simultaneous and backward conditioning. According to the TCH, an association is formed between the CS and the US with each of the four different temporal arrangements, and the specific temporal relationship is encoded as part of the resultant association. In both simultaneous and backward conditioning, the specific temporal relationships between the CS and US do not give the CS any predictive value; hence, the CS fails to elicit anticipatory responding in either case. Miller and colleagues demonstrated the validity of this view by showing that rats can encode interval relationships in simultaneous conditioning (e.g., Barnet, Arnold, & Miller, 1991) and backward conditioning (e.g., Arcediano, Escobar, & Miller, 2003; Molet, Miguez, Cham, & Miller, 2012) as well as trace conditioning with relatively long interstimulus intervals (e.g., Cole, Barnet, & Miller, 1995). These researchers used sensory preconditioning (i.e., S2-S1 trials followed by S1-US trials, resulting in conditioned responding to S2) and second-order conditioning (i.e., S1-US trials followed by S2-S1 trials, resulting in responding to S2) procedures modified to reveal these three elusive varieties of conditioning (see Figure 1). Basically, the TCH posits that the so called simultaneous and backward conditioning deficits are not deficits in forming associations; the associations are formed but do not support anticipatory responding, which is what is assessed in most laboratory conditioning preparations. Alternatively stated, the so called simultaneous and backward conditioning deficits are not associative deficits, but performance deficits. Presented below are a few experiments supporting these conclusions.

Figure 1.

(A) Trace conditioning: The experimental condition, hypothetical temporal representations, and observed result of Cole, Barnet, and Miller (1995, Experiment 1). (B) Simultaneous conditioning: The experimental condition, hypothetical temporal representations, and observed result of Barnet, Arnold, and Miller (1991). (C) Backward conditioning: The experimental condition, hypothetical temporal representations, and observed result of Molet, Miguez, Cham, and Miller (2012, Experiment 2). Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus. CR = strong conditioned responding (i.e., suppression).

For the sake of clarity, we will describe only the critical experimental group for each demonstration, all of which used different versions of second-order conditioning to reveal learning that would often have been latent if we had assessed only responding to the first-order CS (i.e., S1; see the left side of Figure 1). Cole et al. (1995; see Figure 1A) gave first-order S1-US pairings with a 5-s gap (i.e., trace interval), followed in Phase 2 by S1-S2 pairings in a backward fashion (i.e., S1 just before S2). Counterintuitively, they observed a stronger CR to the second-order stimulus (S2) than the first-order stimulus (S1). Barnet et al. (1991; see Figure 1B) exposed rats to simultaneous presentations of S1-US (5-s duration); in Phase 2 they were exposed to 5-s presentations of S2, each of which terminated with the onset of a 5-s presentation of S1. At test, strong responding to S2 was observed. Molet et al. (2012; see Figure 1C), adapting the procedure of Arcediano et al. (2003) to rats, administered S1-footshock US pairings in a backward relationship (i.e., US-S1) with a 4-s gap (i.e., trace interval) between termination of the US and onset of S1 in Phase 1, followed by S2-S1 pairings in a forward relationship with a 5-s gap between termination of S2 and onset of S1 in Phase 2. When tested on S2, rats exhibited a robust CR. The right side of Figure 1 depicts the hypothetical integrated temporal maps following each experimental situation.

According to the TCH, the rats had encoded the temporal relationships between S1 and the US and between S1 and S2, thereby forming two independent temporal maps, which included the order and interval between the paired events. These temporal maps presumably were integrated by superimposing the representation of the common element from the two phases of training (i.e., S1), thereby allowing S2 to predict an impending US when rats were tested, a relationship that is conducive to conditioned responding to S2.

Taken together, these findings support the view that the temporal relationships among events during training are encoded as attributes of the association. Additionally, this series of experiments showed that when subjects independently acquire two associations with a common element (e.g., S1-US and S2-S1), each with its own temporal relationship, they behave as if the two unique cues have a known temporal relationship despite their never having been paired. Seemingly, they have integrated the two associations to create a third association with its own temporal relationship (S2-US). This tenet is what makes the TCH unique in comparison to real-time models (e.g., Church & Broadbent, 1990; Machado, 1997; Staddon & Higa, 1999; Sutton & Barto, 1990; Vogel, Brandon, & Wagner, 2003; Wagner, 1981). This is particularly evident in the case of temporal integration involving a backward association (e.g., Arcediano et al., 2003; Molet et al., 2012). Indeed, it is hypothesized that subjects encoded the specific intervals from the US to S1 and from S2 to S1, and that they were able to retrieve the backward temporal location of the US with respect to S1 when S2 was presented at test. This challenges the view adopted by most timing models that measurement of time is always in the forward direction, because there is now good evidence that upon presentation of a cue animals can retrieve from memory a representation of an event that previously occurred immediately prior to earlier presentations of the cue (i.e., backward associations are encoded).

It is worth noting that second-order conditioning and sensory preconditioning are themselves problematic for simple contiguity theory, which posits that contiguity between the CS and US is essential for the establishment of a CS-US association. In both second-order conditioning and sensory preconditioning, the target cue (X) is never presented in close contiguity with the US. Hence, the contiguity principle does not predict responding to X. Yet, X comes to elicit responding in both situations. As we have seen above, the TCH accounts for such responding through its fourth tenet, which posits integration of independently acquired temporal relationships that contain a common element.

3. Cue competition

Abundant research has been devoted to understanding the mechanisms of cue competition, which refers to impaired responding to a CS (X) when it has been paired with a US in the presence of a second CS (A) that is either highly salient (i.e., overshadowing) or has previously been paired with the US by itself (i.e., blocking). Cue competition further challenges the contiguity principle because the repeated associations between a CS and a US do not necessarily lead to the elicitation of a CR. For example in overshadowing treatment (i.e., AX-US pairings), the overshadowed cue (X) has the same temporal relationship with the US as in the conventional control group for overshadowing (i.e., X-US training), but X in the overshadowing group subsequently elicits less responding than in the control group despite identical contiguity of X with the US in the two conditions. In recent decades, considerable effort has been devoted to determining whether cue competition results from a deficit in learning or performance. Importantly, post-training treatments have proven to be a useful strategy to nourish this debate. For example, post-training extinction of the overshadowing (or blocking) cue has been found to increase responding to the overshadowed or blocked cue (i.e., retrospective revaluation; e.g., Blaisdell, Gunther, & Miller, 1999; Kaufman & Bolles, 1981; Matzel, Schachtman, & Miller, 1985). Not surprisingly, associative theories designed to explain cue competition (e.g., Mackintosh, 1975; Miller & Matzel, 1988; Pearce & Hall, 1980; Rescorla & Wagner, 1972) can account for the basic phenomena of cue competition (e.g., blocking, overshadowing); however, they do not assign any role to temporal content in conditioned associations. The same is also true for most of the modern versions of these theories that explain retrospective revaluation effects following cue competition (e.g., Aitken & Dickinson, 2005; Stout & Miller, 2007; Van Hamme & Wasserman, 1994).

In summary, despite their theoretical disparities in regard to whether cue competition is a deficit of acquisition or performance, these models share the view that good temporal contiguity facilitates the development of an association but does not become part of that association. Here we will present some recent evidence suggesting that this perspective is not supported by data, and that theories of conditioning need to include time as part of associations in order to fully account for cue competition.

Using a blocking procedure, Amundson and Miller (2008) demonstrated that rats encode temporal relationships between events during conditioning (see Figure 2A). When X and A were trained with equally long trace intervals in Phase 2 (i.e., 15-s), less blocking resulted from A being trained in Phase 1 with a short trace interval (i.e., 3-s) than with a long trace interval similar to the interval used in Phase 2 (i.e., 15-s). This finding is noteworthy because it is inconsistent with most associative models (which predict that short delay conditioning in Phase 1 would result in a more robust A-US association that should yield more blocking of X during Phase 2). However, it is consistent with the TCH, which anticipates that X’s relationship with the US is redundant with A’s relationship with the US only when X and A have similar interstimulus intervals with the US. Amundson and Miller’s observations are consistent with two complementary assertions: (1) that the temporal interval between a CS and a US constitutes one of the attributes encoded in an association, as posited by the TCH, and (2) that cue competition is maximal when the competing cues convey the same information, as proposed by Miller and Matzel (1988). According to Miller and Matzel’s comparator hypothesis model, conditioned responding to the target CS is directly related to the magnitude of the US representation that is directly activated by the target CS and is inversely related to the magnitude of the US representation that is indirectly activated by the target CS (i.e., mediated by the target CS’s comparator stimulus through conjoint action of the target CS-comparator stimulus and comparator stimulus-US associations). At test, conditioned responding is assumed to reflect a comparison of the US representations directly and indirectly activated by the target CS, with strong CRs when the competitor’s associative strength with the US is weak and with weak CRs when the competitor’s associative strength with the US is strong. Appling the comparator hypothesis to blocking, Amundson and Miller’s blocked CS was trained in Phase 2 with a long interval between the onset of the CS and the onset of the US. Given the target CS-blocking CS association that was acquired in Phase 2, the group with the long interval in Phase 1 between the blocking CS (i.e., comparator stimulus) and the US (i.e., Group Blocking Same), upon test with the target CS, had an indirectly activated expectation of the US that matched in time the expectation of the US directly activated by the target CS, in contrast with the group trained with a shorter interval in Phase 1 (Blocking Different Group), thereby resulting in attenuated conditioned responding to CS X only in Blocking Same Group. Paralleling the findings of Amundson and Miller, Blaisdell, Denniston, and Miller (1998) demonstrated that overshadowing is maximal when the two competing cues have the same temporal relationship with the US, which strongly suggests that the interstimulus interval is encoded as part of the CS-US association. The same account of the role of CS-US intervals in blocking presented by Amundson and Miller can be applied to the overshadowing data of Blaisdell et al. (1998).

Figure 2.

(A) The experimental design and observed results of Amundson and Miller (2008, Experiment 3). Group Blocking Different assessed whether temporal information per se is encoded during training within a blocking procedure. (B) The experimental design and observed results of Blaisdell, Denniston, and Miller (1999, Experiment 1). Group Attenuation of Overshadowing assessed whether temporal information per se is encoded during retrospective revaluation training. Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus. CR = strong conditioned responding; Cr = medium conditioned responding; cr = little or no conditioned responding.

Blaisdell, Denniston, and Miller (1999) assessed the role of temporal relationships in retrospective revaluation (see Figure 2B). In their Experiment 1, they administered overshadowing treatment with a trace relationship between the competing cues (A and X) and the US in Phase 1, followed by temporal updating training of A (temporal revaluation) with a trace (same temporal relationship as in prior overshadowing treatment) or delay (different temporal relationship) situation with the US in Phase 2. As expected in the conjoint framework of Miller and Matzel’s comparator hypothesis model and the TCH, dissimilar A-US intervals in Phases 1 and 2 reduced overshadowing of X relative to the condition for which the A-US interval remained unchanged between Phases 1 and 2. According to the comparator hypothesis model, posttraining associative inflation of the comparator stimulus (A) with a different temporal relationship with the US (compared to Phase 1) reduced the overshadowing stimulus’ potential to compete with the overshadowed CS (X) at test, given that the A-US temporal relationship was now different from the X-US temporal relationship. Importantly, this observation is not explicable in terms of backward blocking without temporal encoding because the longer interstimulus interval of Phase 2 should then result in a weaker association and consequently less backward blocking, which is contrary to what was observed.

4. Conditioned inhibition

Conditioned inhibition is another hallmark phenomenon in classical conditioning that has captured the attention of students of learning since the pioneering work of Pavlov (1927). In Pavlov’s conditioned inhibition procedure, a CS (A) is made excitatory as a result of repeated pairings with the US (A-US), and on interspersed trials a second cue (X) is paired with this excitatory CS in the absence of the US (AX-). With sufficient AX- trials, this treatment makes X function as a conditioned inhibitor capable of passing both summation and retardation tests for conditioned inhibition (Rescorla, 1969).

The prior discussion makes clear that subjects learn from a cue (e.g., a CS) not only that an outcome (e.g., a US) will occur, but when it will occur. When considering the role of temporal relationships in conditioned inhibition, the question of interest is whether a conditioned inhibitor informs subjects of exactly when a previously expected US will be omitted. This question is fundamental in the sense that it provides a more general answer to the question of whether associations, both excitatory and inhibitory, encode temporal information. Applied to conditioned inhibition, the comparator hypothesis posits that inhibitor X generates a negative response potential when it activates a US representation through its comparator stimulus A that is stronger than the US representation directly activated by X. In other words, because the X-A and A-US associations are strong relative to the X-US association, the negative response potential of X should be high. Consistent with the effects of temporal similarity on cue competition (e.g., Amundson & Miller, 2008), Miller and colleagues found that, on a summation test during which a conditioned inhibitor is compounded with a transfer excitor, the inhibitor exerts maximal inhibitory behavioral control when its temporal relationship to the US omitted during inhibitory training corresponds to the time that the US is expected based on the transfer CS (e.g., Barnet & Miller, 1996; Denniston, Cole, & Miller, 1998). Complementing these timing effects on summation tests for conditioned inhibition, Burger, Denniston, and Miller (2001) examined timing in retardation tests for inhibition. They found that a conditioned inhibitor, when paired with the US during a retardation test, requires more pairings in order to elicit a CR when its temporal relationship to the US that was omitted during inhibitory training is equal to the temporal relationship of the inhibitor to the US during the inhibitor-US pairings of the retardation test. Thus, there is evidence from both summation and retardation tests that a conditioned inhibitor not only signals that a US will be omitted, but also when it will be omitted.

Denniston, Blaisdell, and Miller (2004) extended the demonstration of temporal relationships being encoded during Pavlovian conditioned inhibition training (i.e., A-US / AX- treatment) through posttraining manipulation of the A-US interstimulus interval (See Figure 3). In Denniston et al.’s study, rats received treatment in which A was trained as a delay excitor with no temporal gap between termination of A and onset of the US, and X was trained as a simultaneous inhibitor with A (i.e., X and A were presented with common onset and termination). In preparation for subsequent summation tests, one transfer excitor (C) was trained as a delay transfer CS with no temporal gap between CS termination and US onset, and a second transfer excitor (D) was trained as a trace transfer CS with a 5-s gap between CS termination and US onset. Following this training, half of the subjects received posttraining revaluation of the A-US temporal relationship consisting of A-US pairings, now with a 5-s gap between CS termination and US onset. The remaining subjects received the same training except that A was replaced by B (where B was an irrelevant stimulus used for control purposes). During the summation tests, subjects that were trained with B_US pairings during posttraining treatment exhibited greater negative summation indicative of inhibition when tested with the XC stimulus compound than with the XD stimulus compound (see Figure 3B). In contrast, the subjects in the temporal revaluation group that were trained with the new temporal relationship between A and the US showed the reverse pattern of results (see Figure 3A). The authors hypothesized that, for subjects for which the A-US interval was not altered, inhibitor X generated a negative expectation of the US at the same temporal location as C but not D because C predicted the occurrence of the US at the same temporal location as X’s training excitor (A; i.e., C and A were trained with no gap between CS termination and US onset). In contrast, for subjects for which the A-US interval was modified, inhibitor X generated a negative expectation of the US at the same temporal location as D but not C because D predicted the occurrence of the US at the same temporal location as X’s training excitor (A; i.e., D and A were most recently trained with a 5-s gap between CS termination and US onset). Denniston et al.’s data represent a form of temporal retrospective revaluation in a conditioned inhibition situation, which mirrors the previously described effect of temporal retrospective revaluation in excitatory conditioning (specifically overshadowing; Blaisdell et al., 1999).

Figure 3.

Experimental conditions and observed results of Denniston, Blaisdell, and Miller (2004, Experiment 1). (A) Negative summation was observed for XD but not for XC when the temporal relationship of A and the US was revalued. (B) However, without revaluation of the A-US temporal relationship, negative summation was observed for XC but not for XD. Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus. CR = strong conditioned responding; cr = little or no conditioned responding.

Taken together, these outcomes lend further support to the TCH assumption that the temporal information is inevitably encoded in inhibitory associations as well as excitatory associations. Importantly, as noted by Miller and colleagues, the TCH alone is not sufficient in and of itself because it has no mechanism for explaining conditioned inhibition per se. However, integration of the TCH and Miller and Matzel’s comparator hypothesis for interaction of cues trained in compound provides a satisfactory explanation of the role of temporal information in conditioned inhibition.

5. Associative Interference

Associative interference is another situation in which it has been demonstrated that associations include not only the physical attributes of the paired stimuli, which provide a means by which presentation of one of them will activate the representation of the other, but their individual and interstimulus temporal characteristics as well (i.e., stimulus durations and interstimulus interval). Consider retroactive cue interference. Here subjects trained on X-Outcome (O) followed by A-O trials commonly exhibit less recall of X-O association during testing than subjects lacking the A-O experience. Extending the postulates of the TCH, Miller and colleagues (e.g., Escobar & Miller, 2003) proposed that associative interference should be maximal between cues that hold a similar temporal relationship to the outcome, and conversely that interference should be reduced between cues that convey dissimilar temporal relationships to the outcome. This is precisely what Escobar and Miller (2003) found using the X-O, A-O paradigm embedded within a sensory preconditioning preparation. That is, they observed greater retroactive interference in responding to X (1) when the X-O and A-O interstimulus intervals were similar (see Figure 4A), (2) when the durations of the target CS (X) and the interfering CS (A) were similar (see Figure 4B), and (3) when the durations of the outcome were identical during separate phases of training with X and A (see Figure 4C). Miller and Escobar’s (2002) similarity principle accounts for Escobar and Miller’s (2003) results assuming that associations that are more similar to each other in all aspects, including temporal structure, are more apt to interact (at least until similarity is so great that Phase 2 training is simply more of the same training received in Phase 1; then of course one often sees facilitation due to more training rather than interference). This of course implies that temporal information is encoded as part of each association, as postulated by the TCH. In the situations in which responding to the target CS (X) was strongly reduced, presumably the A-O association was primed due to its recency of acquisition, thereby allowing it to interfere with retrieval of the X-O association when the two associations shared their temporal relationships and outcome. In the cases in which the retroactive interference was not observed (i.e., strong responding to CS X), we hypothesize that there was less interference because the two associations were more dissimilar in their temporal attributes.

Figure 4.

(A) Experimental design and observed results of Escobar and Miller (2003, Experiment 1). Group Diff. intervals assessed whether temporal intervals between cues encoded during training affected retroactive interference. (B) Experimental design and observed results of Escobar and Miller (2003, Experiment 2). Group Diff. X & A assessed whether cue durations encoded during training affected retroactive interference. (C) Experimental design and observed results of Escobar and Miller (2003, Experiment 3). Group Diff. O assessed whether outcome durations encoded during training affected retroactive interference Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus. CR = strong conditioned responding; cr = little or no conditioned responding.

Escobar and Miller’s (2003) study focused on the role of temporal relationships in cue interference which occurs when two CSs are independently trained with the same outcome. Another example of temporal encoding is provided in situations in which a single CS is paired with a single outcome with two different interstimulus intervals, each in a different context. Molet, Urcelay, Miguez, and Miller (2010) provide evidence that both interstimulus intervals are encoded and that subjects use relative similarity of the test context to each of the training contexts to determine which interstimulus interval to express at test. In the first phase of a sensory preconditioning procedure, rats were exposed to pairings of two cues, S1 and S2, in two distinctly different contexts, X and Y. The S1-S2 interstimulus interval differed between Contexts X and Y. In the second phase of the procedure, the rats were exposed to S1-US pairings (with a constant interstimulus interval) in both Contexts X and Y. Testing on S2 in each of the two contexts found that the test context (i.e., X or Y) determined which interval relationship from Phase 1 was integrated with the S1-US interval relationship acquired in Phase 2, thereby allowing for the expression of two different interval relationships between S2 and the US as a function of the test context. Based on the TCH, we assumed that the encoding of two conflicting interval relationships between S1 and S2 created ambiguity regarding the relative temporal locations of those cues with respect to each other. The information acquired in the training context that was most similar to the test context seemingly was primed for retrieval by the test context, which is consistent with Miller and Escobar’s (2002) model of interference.

6. Models of temporal encoding

In contrast to most associative theories, models of timing propose mechanisms by which animals perceive and encode temporal intervals between events and by which a cue can serve as a time marker indicating the start of a timed interval (e.g., Church & Broadbent, 1990; Machado, 1997; Staddon & Higa, 1999). The timing models generally fail to adequately account for cue competition, conditioned inhibition, and associative interference. Conversely, although the associative models do explain basic cue competition, conditioned inhibition, and sometimes associative interference, they do not address the role of temporal information in these phenomena as described earlier in this review. There are a few real-time associative models that do begin to account for timing effects within these phenomena (e.g., Sutton & Barto, 1990; Vogel, Brandon, & Wagner, 2003), but none of them anticipate integration of temporal relationships between two different associations sharing a common stimulus (e.g., Barnet & Miller, 1996; Matzel, Held, & Miller, 1988). Therefore, we are left with two families of models (i.e., models of associative learning and models of timing), each with their own limited domain of explicable behavioral phenomena.

Straddling these two families of models is the TCH, which addresses how temporal information is used in generating Pavlovian responses, including temporal integration. However, the TCH by itself is not a model of how time is perceived, nor is it a complete model of associative learning. Although the TCH can explain the effects of temporal encoding on all of the conditioning effects discussed above, alone it is unable to explain basic cue competition, conditioned inhibition, and associative interference. However, because the TCH posits that contiguity alone is sufficient for the formation of associations, it pairs well with Miller and Matzel’s (1988) comparator hypothesis which also assumes that contiguity alone is sufficient for learning to occur and that many learned associations are behaviorally silent (see Amundson & Miller, 2008; Blaisdell et al., 1999; Denniston et al., 1998, for specific examples of how the comparator hypothesis and the TCH conjointly are able to account for many cue competition and conditioned inhibition effects including the influence of encoded temporal relationships). The TCH also pairs well with Miller and Escobar’s (1992) retrieval model, which conjointly are able to explain associative interference effects including the role of encoded temporal relationships (e.g., Escobar & Miller, 2003; Molet et al., 2010). The TCH does not subscribe to any specific mechanism of timing, but is compatible with several timing models. At this time, it would be premature to favor any specific timing account; various models of timing should be entertained. To adequately embrace the role of time in associative learning, it is likely that real time associative models will ultimately be required. But we do not think progress will be made by entertaining models that encompass so many different cognitive processes and parameters that unambiguous a priori testable predictions cannot be generated (e.g., Buhusi & Schmajuk, 1999; Schmajuk, Lam, & Gray, 1996). The TCH makes clear testable a priori predictions, many of which have been born out. However, the TCH has not been formalized in mathematical terms; that is, it makes only qualitative predictions (i.e., rank ordering of responding among conditions).

7. Temporal integration at the time of testing

As described above, it is now clear that when two independent temporal relationships (e.g., S1-S2 and S1-US) are learned and have a common element (here S1), they are integrated through superpositioning of the two representations of the common element, thereby creating an effective temporal relationship between the two unique elements (S2 and the US in the present case). However, until recently, it was unclear “when” temporal integration occurs. One possibility is that temporal integration occurs at the time that the second temporal relationship is learned. An alternative possibility is that temporal integration occurs at the time of testing. To investigate when temporal integration occurs, Molet et al. (2012) used Arcediano et al.’s (2003) preparation in both sensory preconditioning and second-order conditioning procedures and examined the effect of extinguishing S1 prior to testing on S2. Figure 5 summarizes the critical experimental groups with the hypothetical temporal maps assuming integration occurred during acquisition of the second association or at test. We observed that extinction of S1 (i.e., extinction of the S1-S2 and US-S1 associations) prior to testing disrupts responding (i.e., conditioned suppression) to S2, and we interpreted this as indicating that temporal integration occurs at the time of testing rather than at the time of training. If temporal integration has occurred during acquisition of the second association (i.e., an S1-US association), subsequent extinction of S1 should have had no consequence on responding to S2.

Figure 5.

(A) The sensory preconditioning experimental design, hypothetical temporal representations, and observed results of Molet, Miguez, Cham, and Miller (2012, Experiment 1). (B) The second-order conditioning experimental design, hypothetical temporal representations, and observed result of Molet, Miguez, Cham, and Miller (2012, Experiment 2). In A and B, extinction of S2 during Phase 3 assessed whether integration occurred during Phase 2 training (i.e., before extinction of S2) or at the time of test (i.e., after extinction of S2). Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus; each dashed line represents the retrieved activation of a representation. CR = strong conditioned responding; cr = little or no conditioned responding.

Based on the view that temporal integration occurs at test, the rats should have encoded two associations concerning S1, each with its own temporal relationship (S1-S2 and US-S1), which would conflict with the memory established during Phase 3 of S1 alone (i.e., extinction memory of S1). When subjects acquire conflicting information in a phasic manner (i.e., pairings of S1 with S2 or the US in Phases 1 and 2 followed by extinction of S1 during Phase 3), a short retention interval before testing is apt to yield behavior consistent with the most recently acquired information (i.e., extinction memory of S1, a recency effect). Accordingly, this should have resulted in a decrease in integration of the S1-S2 and the US-S1 temporal relationships at test in both the sensory preconditioning and second-order conditioning preparations. In others words, due to the recency of the extinction memory of S1, it was less likely that at test S2 would activate a representation of S1 that would in turn activate a representation of the US (see Figure 5A). This was consistent with our observation of little conditioned responding (i.e., suppression) to S2.

The view that temporal integration occurs at the moment of Phase 2 training predicts a strong conditioned responding to S2. Based on this account, the rats should have encoded the temporal relationship between S1 and S2 (including order and interval) during Phase 1. When S1 was backward paired with the US during Phase 2, the rats should have retrieved a representation of S2 having preceded S1, retrospectively making the S2 representation immediately antecedent to the US of Phase 2. This should have allowed the subjects to encode a temporal relationship between the representation of S2 and the representation of the US during Phase 2, with the US occurring immediately after termination of S2. At test, onset of S2 should have activated a representation of the US with its onset shortly thereafter. Consequently, rats should have expected the US to be delivered immediately after termination of S2, which would be conducive to robust conditioned responding (see Figure 5B).

Molet et al. (2012) thus established that integration occurs during presentation of the target cue (S2) at testing, at least with the parameters that they used. It was important to determine when integration of temporal relationships occurs because this knowledge speaks to associative structure during the retention period between acquisition of second association and testing. Moreover, this finding adds to the common ground between the TCH and both Miller and Matzel’s (1988) comparator hypothesis, which posits that cue competition and conditioned inhibition arise from associations that are compared at the time of testing, and Miller and Escobar’s (2002) account of associative interference, which posits that interference is the result of competition between associations that occurs at test. That is, the TCH, Miller and Escobar’s account of interference, and the comparator hypothesis all emphasize the processing of information that occurs at test, rather than assume that all critical information processing occurs during training and that there is minimal processing at test. We believe that time has come for models of learning based on data from nonhuman subjects to show more concern for processes in play during a test trial.

8. Associative structure

Molet et al. (2012) established that presentation of S2 at test causes the subject to expect S1 x seconds in the future and consequently expect the US y seconds in future (where x is greater than y), and that it is this expectation of the US in the immediate future that produces robust responding to S2. In a further effort to illuminate the mechanism of temporal integration, Polack, Molet, Miguez, and Miller (2013) conducted a study designed to assess associative structure following an initial test trial. Figure 6 depicts the important experimental groups along with the different hypothetical associative structures. More specifically, there are two possible associative structures that could exist following an initial test on which temporal integration occurs: (1) Conditioned responding to S2 on subsequent tests could be the result of recurring integration of the two independently learned temporal maps that remain independently stored in memory (i.e., S1-S2 plus US-S1); (2) Temporal integration at the moment of initial testing could result in the formation of a direct S2-US (or S2-CR) association.

Figure 6.

The experimental design, hypothetical temporal representations, and observed result of Polack, Molet, Miguez, and Miller (2013, Experiment 1). (A) Group Ext assessed whether integration resulted in a S1-US associative structure or could be the result of two independently learned interval maps that remain independently stored in memory. (B) Group Cont controlled for temporal integration. Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus; each dashed line represents the retrieved activation of a representation. CR = strong conditioned responding.

Polack et al. (2013) sought to differentiate between these two structures by extinguishing S1 after observing evidence that integration has occurred on an initial test of S2. The critical experiment was designed to replicate the findings of Molet et al. (2012) using a slightly different procedure to determine whether an initial test of S2 was sufficient to generate a new S2-US (or S2-CR) association that was no longer dependent on the associative status of S1. On the second test of S2, if conditioned responding were due to integration during the first test that resulted in a S2-US (or S2-CR) temporal map, then this responding should be insensitive to extinction of S1 after integration has occurred on an initial test trial. In contrast, if extinction of S1 reduces subsequent responding to S2 after an initial S2 test, this would suggest that integration occurs repeatedly on each test trial, each integration based on a chain of temporal maps, S1-S2-US (where the temporal relationship between S2 and the US is backward), rather than the formation of a new direct temporal map, S2-US (or S2-CR). As Molet et al. (2012) argued, once the formation of a S2-US map has occurred, it should be insensitive to extinction of S1.

The results of Polack et al. (2013) confirmed previous findings that extinction of S1 prior to testing S2 prevented integration from being observed (i.e., little conditioned responding was observed). However, when extinction of S1 was delayed until after an initial test of S2, this same extinction treatment was ineffective at attenuating integration. That is, responding was just as robust as if no extinction of S1 had occurred. These findings support the interpretation that temporal integration at the moment of testing takes the form of a new S2-US (or S2-CR) association. Responding after integration has occurred appears to be entirely independent of the associative status of S1.

9. Bidirectionality of associations

Early work in our laboratory demonstrated that in addition to conventional forward associations from antecedent stimuli to subsequent stimuli being formed by contiguous stimulus presentations, backward associations were also formed from subsequent stimuli to antecedent stimuli (e.g., Matzel et al., 1988). But what was not resolved was whether independent forward and backward associations were formed or a single bidirectional association was established. Arcediano, Escobar, and Miller (2005) addressed this question. That is, when subjects are presented with a pair of sequential stimuli, S2 followed by S1, there are two types of associations that could plausibly be formed: a single bidirectional association (i.e., S2↔S1) or two unidirectional associations (i.e., S2→S1 and S2←S1). Which of these associative arrangements resulted was assessed by extinguishing S1 (or S2) after S2→S1 pairings and before US→S1 pairings. Here we report only the three critical conditions that allowed the researchers to conclude that the association between two events was based on a single bidirectional link (see Figure 7).

Figure 7.

The experimental design, hypothetical temporal representations, and observed result of Arcediano, Escobar, and Miller (2005, Experiment 2). (A and B) Groups Ext-S1 and Ext-S2 assessed whether integration was based on two unidirectional links or one bidirectional link. (C) Group Ext-S3 controlled for temporal integration. Open rectangles and letters represent conditioned stimuli; black rectangles represent the footshock unconditioned stimulus; each dashed line represents the retrieved activation of a representation. CR = strong conditioned responding; cr = little or no conditioned responding.

In Arcediano et al.’s study (2005), rats were administered S2-S1 pairings in a forward relationship with a 5-s gap (i.e., trace interval) between termination of S2 and onset of S1 in Phase 1. During Phase 2, Group Ext-S1 received presentations of S1 alone (i.e., extinction of S1), whereas Group Ext-S2 received presentations of S2 alone (i.e., extinction of S2). In Phase 3, all rats were exposed to S1-footshock US pairings presented in a backward relationship (US-S1) with a 4-s gap between termination of the US and onset of S1. (Control groups in Arcediano et al.’s experiment demonstrated that latent inhibition of the US-S2 association of Phase 3 due to Phase 2 exposure to S2 alone was not responsible for the weak responding observed to S1 in Group Ext-S2.) Group Ext- S3 demonstrated that, without extinction of S1 or S2, Phases 1 and 3 supported conditioned responding. As predicted uniquely by the view that the two stimuli were linked by a single bidirectional link, Arcediano et al. observed no conditioned responding to S2 in either Group Ext-S1 or Group Ext- S2 presumably because presentations of S1 alone to Group Ext-S1 during Phase 2 extinguished the S2-S1 association to the same degree as the S2 alone presentations given to Group Ext-S2. In those circumstances, it was not possible to accomplish the temporal integration needed to support responding to S2 because S2 could not activate a representation of S1. These results were contrary to the view that the stimuli were linked by two unidirectional links. Given unidirectional associations, extinction of S2 in Group Ext-S2 should have weakened the S2→S1 association, whereas presentation of S1 alone in Group Ext-S1 should have only weakened the S2←S1 association but not the S2→S1 association. In this case, temporal integration of the information acquired in Phases 1 and 3 would not be possible in Group Ext-S2 but would be in Group Ext-S1. This view incorrectly predicts a strong conditioned responding to S2 in Group Ext-S1 due to the presumed temporal integration.

10. Implications for associative and temporal learning

Timing of intervals between two events requires that the two events be co-identified. This co-identification constitutes what we define as an association. In principle, an association could be learned without encoding a temporal relationship (as was assumed by Aristotle and the human associative learning researchers of the 1940–1960s), but the data reviewed here do not favor this view. Seemingly, every perceived pair of events results in the encoding of the temporal relationship between the events (i.e., when).

Gallistel subscribes to temporal relationships, without any other associative attributes, being learned and views them as sufficient to account for all so-called associative phenomena (e.g., Gallistel & Gibbon, 2001). But we think that timing between two events has no meaning without coencoding the two events because intervals must have beginning and end markers. This co-encoding is what we define as an association. As important a role as temporal information has, there also appears to be important nontemporal information which is encoded as part of an association such as spatial information (e.g., Amundson & Miller, 2007; Sawa, Leising, & Blaisdell, 2005) and the magnitude of a reinforcer (e.g., Morris & Bouton, 2006). Furthermore, we question whether co-encoding occurs between events that are very far apart in time (i.e., weeks, months, and years), which is assumed by Gallistel. Although our timing data suggest that the weak conditioned responding observed when there is a relatively long interval between the two events (i.e., poor contiguity) often reflects a performance deficit (i.e., a failure to express an association rather than a failure to encode it), we suspect that no association between events is formed when there is a sufficiently long interval between the two events.

Associative learning (or at least associative performance) is dependent on timing, in that it requires temporal proximity between cue and outcome, which is equivalent to temporal contiguity in the broad sense (as opposed to the narrow sense which demands simultaneity). This dependency could be passive in that it could arise from spontaneous decay of the cue representation prior to the outcome rather than the cue-outcome interval being actively timed. But critically, delayed outcomes often result in delayed responses instead of (or in addition to) weaker responses.Moreover, even if this dependence on contiguity arose from the passive decay of cue representations, passive decay is a timing mechanism of sorts. Thus, associative learning surely depends on some form of timing. However, timing alone cannot fully account for associative learning as there are nontemporal variables that influence learning which cannot be reduced to temporal information. Moreover, identical training procedures are known to simultaneously produce several different types of conditioning including sensory, hedonic (e.g., Wagner & Brandon, 1989), motivational (e.g., Timberlake, 2001), and response attributes (e.g., Cabrera, Sanabria, Jiménez, & Covarrabias, 2013). Although these attributes are simultaneously conditioned with identical temporal relationships of stimuli, they develop at somewhat different rates and to different asymptotes. These differences further speak to the importance of learning variables beyond temporal relationships. Thus, timing and associative learning are integrally related, but neither can be fully reduced to the other.

We adopt the view that associations between a cue (i.e., a CS) and an outcome (i.e., a US) have multiple attributes. That is, the cue comes to activate assorted outcome attributes including sensory, hedonic (i.e., affective), motivational, and response (i.e., affordances) attributes. An early example of theoretically distinguishing between sensory and hedonic attributes of an association is provided by Wagner’s AESOP (Wagner & Brandon, 1989; also see Konorski, 1967, for a yet earlier version of this, and Delamater, 2007, for a more recent version). In principle, these different US attributes could be inseparable, which would make differentiating among them unnecessary. But considerable data suggests that the different attributes are sometimes processed independently of each other (see Delamater, 2007, 2012a, 2012b). Importantly, although sensory, hedonic, motivational, and response attributes are separable from one another, each of these attributes appear to contain elements of where and possibly when.

Highlights.

• Associations appear to always include temporal information about the associated events.

• Temporal information without events as anchors does not exist.

• Associative models and timing models complement each other in accounting for acquired behavior.

• There is a need to better integrate associative theories with models of timing.