Reward magnitude effects on temporal discrimination

Abstract

Changes in reward magnitude or value have been reported to produce effects on timing behavior, which have been attributed to changes in the speed of an internal pacemaker in some instances and to attentional factors in other cases. The present experiments therefore aimed to clarify the effects of reward magnitude on timing processes. In Experiment 1, rats were trained to discriminate a short (2 s) vs. a long (8 s) signal followed by testing with intermediate durations. Then, the reward on short or long trials was increased from 1 to 4 pellets in separate groups. Experiment 2 measured the effect of different reward magnitudes associated with the short vs. long signals throughout training. Finally, Experiment 3 controlled for satiety effects during the reward magnitude manipulation phase. A general flattening of the psychophysical function was evident in all three experiments, suggesting that unequal reward magnitudes may disrupt attention to duration.

The effect of motivation on timing was first reported by Roberts (1981) in a peak procedure. The peak procedure involves training with a mixture of reinforced fixed interval trials where food is available after a certain fixed time from signal onset, and peak trials where the signal is presented for longer than normal and food is withheld. When rats were pre-fed prior to testing on a peak procedure, Roberts (1981) reported a rightward shift in the peak and a depression in response rates. More recent studies indicated that chronic decreases in reward magnitude (Ludvig, Conover & Shizgal, 2007) or reward value (through pre-feeding or lithium chloride-induced flavor aversion) also shift the peak to the right (Galtress & Kirkpatrick, 2009), and that chronic increases in reward magnitude produce a leftward shift in the peak procedure (Galtress & Kirkpatrick, 2009; Grace & Nevin, 2000; Kacelnik & Brunner, 2002) and differential reinforcement of low rate (DRL) procedure (Doughty & Richards, 2002).

The effects of reward magnitude and/or value on responding in the peak and DRL procedures are significant because they indicate that motivation and timing are not necessarily independent processes. Moreover, these chronic effects of reward parameters on timing are not easily incorporated into current timing models.

Scalar expectancy theory (SET; Gibbon & Church, 1984), the predominant timing model, contains three modules: a pacemaker-accumulator clock process, a reference memory store, and a decision process. The clock process consists of a pacemaker that emits pulses at a variable, rapid rate and an accumulator that receives pulses. Between the two is a switch, which controls the flow of pulses from the pacemaker to the accumulator. When the switch is closed, pulses are then sent to the accumulator. When a reinforcer is delivered, the number of pulses in the accumulator is sent into the reference memory store as a single item, the accumulator value is reset to zero, and the switch opens. The decision process receives input from both the accumulator (which is tracking the current time in the interval) and the reference memory (which contains past memories of previously-reinforced durations) on a continual basis. When the current value of the accumulator comes to more closely approximate a target value that is randomly selected from memory, responding occurs.

Reinforcer magnitude/value changes could potentially affect timing through the SET clock process by altering either the pacemaker rate or the switch operation. For example, it is possible that reward magnitude increases could increase pacemaker rate through increases in arousal. This would result in a leftward shift in the peak because the target number of pulses would accumulate more rapidly with a faster clock. Reward magnitude or value decreases would produce the opposite effect by slowing down the clock. Clock speed alterations have been proposed in the context of SET to explain a multitude of arousal-based effects on timing, including the operation of dopaminergic drugs (e.g., Buhusi & Meck, 2002; Maricq & Church, 1983; Maricq, Roberts & Church, 1981; Matell, Bateson & Meck, 2006; Meck, 1996), temperature effects (see Wearden & Penton-Voak, 1995 for a review), and the effect of arousing stimuli such as click trains (e.g., Wearden, Philpott & Win, 1999).

Alternatively, the effects of reward magnitude/value on timing could emerge from attentional regulation of the switch process. The switch is potentially affected by attention in two ways. One is through the latency to close or open (Gibbon & Church, 1984). For example, poor attention due to decreases in reward magnitude could result in a slower reaction to the onset and/or offset of the signal and this could result in delays in the onset of timing, which would shift the peak to the right by a constant amount of time. The opposite could occur with increases in attention under reward magnitude increases. In addition, once the switch is closed, it can fluctuate throughout the trial, with higher rates of fluctuation associated with poorer attention (Fortin, 2003; Fortin & Massé, 2000; Thomas & Weaver, 1975; Zakay, 1989). Increases (or decreases) in switch fluctuation would reduce (or increase) the rate of pulse accumulation in a similar manner to decreasing (or increasing) clock speed.

Thus, it seems that both attentional and clock effects could produce the observed changes in timing in the peak procedure. Differentiating these two possibilities is difficult because clock speed and attention produce similar effects on peak procedure performance.

An alternative approach to resolving the issue is to use a temporal discrimination task (e.g., Church & Deluty, 1977) which involves training with two different signal durations that are paired with different instrumental responses, with a correct response resulting in food delivery. Usually non-reinforced intermediate durations are then tested resulting in a psychophysical function relating signal duration to the proportion of choices for the longer signal duration. The temporal discrimination procedure can present an excellent alternative to the peak procedure for differentiating between clock speed and attentional effects on timing because changes in clock speed would shift the psychophysical function leftward or rightward, whereas changes in attention would change the slope of the psychophysical function.

Ward and Odum (2006) examined the effect of pre-feeding on temporal discrimination performance. They found that pre-feeding produced a flattening of the psychophysical function consistent with a loss of stimulus control. They argued that this effect may have been due to a decrease in clock speed within the context of the Behavioral theory of timing (BeT; Killeen & Fetterman, 1988). In a further set of experiments, Ward and Odum (2007) examined a variety of potential disruptors, including pre-feeding. Pre-feeding again flattened the psychophysical function, as did other disruptors, and produced a particularly strong disruption in discrimination of the longer samples (a choose-short effect). They concluded that the effects of disruptors were to reduce motivation to attend to the samples, thereby disrupting overall stimulus control. McClure, Saulsgiver, and Wynne (2009) also found an effect of pre-feeding on temporal discrimination. However in this instance a systematic rightward shift in the psychophysical function was produced. This finding is consistent with the effect of pre-feeding on a peak procedure, which shifts the peak to the right (Galtress & Kirkpatrick, 2009; Roberts, 1981). Thus, it seems that even the bisection procedure cannot clearly differentiate between different alternative mechanisms, at least not when an overall reward value change occurs.

The present series of three experiments sought to develop an alternative method for differentiating alternative interpretations of the reward value/magnitude effects on timing using a temporal discrimination task. However, here, the reward magnitude was manipulated for only one of the temporal values in the task to more precisely pinpoint the nature of the effects of the reward manipulations on timing.

Experiment 1

The temporal discrimination task in Experiment 1 was similar to several previous studies (e.g., Maricq & Church, 1983; Maricq et al., 1981; Meck, 1986). Rats discriminated signal durations of 2 and 8 s; they were then tested with 7 geometrically-spaced, non-reinforced durations. Initial training and testing consisted of delivering a single food pellet for correct responses to both signal durations. This amount was then increased to 4 pellets for one of the signal durations only, with the reinforcement on the alternative duration remaining at 1 pellet.

There are a number of alternative predictions that can be derived for the present study. An increase in reward magnitude for one of the signals should increase arousal which should in turn increase clock speed (e.g., Wearden et al., 1999). In the context of SET (Gibbon & Church, 1984), this should shift the psychophysical function to the left. A change in clock speed could also explain McClure et al.’s (2009) findings of a rightward shift in the psychophysical function following a decrease in reward value due to pre-feeding (which should decrease clock speed). On the other hand, BeT (Killeen & Fetterman, 1988) predicts that an increase in reward magnitude should increase the pacemaker rate and this would sharpen the psychophysical function and produce an increase in slope (see Ward & Odum, 2006). The recently introduced behavioral economic model (BEM; Jozefowiez, Staddon & Cerutti, 2009) proposes that there would be a bias towards the option with the higher payoff value, so that if the long sample is associated with 4 pellets, then there would be an overall choose long bias which would shift the psychophysical function upwards (and 4 pellets for the short sample would shift the function downwards). Finally, the increase in reward magnitude could produce a general disruption in attention to the samples, which would flatten the psychophysical function in a similar manner to Ward and Odum’s (2006, 2007) previous findings. The present study will aim to distinguish among these different predictions.

Method

Animals

The animals were 12 male hooded Lister rats (Charles River, UK), approximately 12 weeks old and with a mean ad libitum weight of 351 g (range = 320 – 380 g). Prior to experimental testing, the rats’ weights were reduced to 85% of their original ad lib weight by restricted feeding of 10 g of standard laboratory chow per day (Lab Diet 2002, IPS, UK). During the experiment, the rats were fed a total daily ration of 15 g per day to ensure consistent growth towards their adult weight. The rats were housed in pairs with free access to water in a vivarium that was maintained on a reverse12:12 hr light-dark cycle, with light offset at 4 am. All of the experimental testing was carried out during the dark portion of the cycle.

Apparatus

All phases of the experiment were conducted in a set of 12 operant chambers (Med Associates, Vermont, USA). Each of the 12 chambers measured 25 x 30 x 30 cm and was housed inside of a ventilated, noise attenuating box measuring 74 x 38 x 60 cm. The chambers were located in a single testing room. Each chamber was equipped with a speaker for delivering auditory stimuli, two levers, a houselight, a food cup, and a water bottle. The speaker was located on the right side of the back wall of the chamber, on the opposite wall from the food cup. The houselight was positioned in the top-center of the back wall. Two retractable levers (ENV-122CM) were located on either side of the food cup at approximately one third of the total height of the chamber; lever presses were recorded by a microswitch. A magazine pellet dispenser (ENV-203) delivered 45-mg food pellets (TestDiet MLab rodent tablet) into the food cup. Each head entry into the food cup was transduced by an LED-photocell. The water bottle was mounted outside the chamber; water was available through a metal tube that protruded through a hole in the lower-center of the back wall. Med-PC for windows (Tatham & Zurn, 1989), running on two Pentium III 800-mHz computers (one for each set of six chambers), controlled experimental events and recorded the time of events with a 2-ms resolution.

Procedure

Rats were randomly allocated to one of two groups: 1-4 and 4-1 (n = 6). The group labels indicate the reward magnitude received for correct responses following short and long signals during the reward magnitude manipulation phase.

Pre-training

Pre-training was carried out over three sessions. In the initial session, all rats received magazine training with single food pellets delivered on a variable time (VT) 60-s schedule for 1 hr. The following two sessions consisted of continuous reinforcement (CRF training, with a single food pellet delivered for each lever press on both the left and right levers, one per day (order counterbalanced), for a total of 30 lever presses. Each session lasted a maximum of 2 hr.

Training

All rats received 70 training trials per session, with additional correction trials, until reaching a criterion of 90% correct for three consecutive sessions. Training trials consisted of illumination of the houselight for either 2 or 8 s followed by the insertion of both levers. A correct choice resulted in both levers being withdrawn, delivery of a single food pellet, and the onset of a 15-s intertrial interval (ITI). The assignment of signal durations to left and right levers was counterbalanced across rats. An incorrect choice resulted in the withdrawal of both levers (without food delivery) followed by a 5-s ITI and then a correction trial. Correction trials involved a repeat of the previous incorrect training duration followed by a choice response. Correction trials continued until a correct lever choice was made and food was then delivered.

Testing, baseline

Following training, 20% of trials (14 in total) were non-reinforced probe durations of 2.38, 2.83, 3.36, 4.00, 4.76, 5.66 and 6.73 s. There were two presentations of each duration per session. The other 80% of trials (56 in total) remained as 2-s and 8-s training trials delivering a single food pellet reward, with correction trials following incorrect choices. After each 5-session block of testing, including the final testing block, the rats received retraining sessions identical to the training sessions above to ensure continued accuracy on the training durations. This was continued until the criterion of 90% correct over three days was re-established. A total of 15 test sessions were given.

Testing, reward magnitude manipulation

In the reward magnitude manipulation phase, the reward for correct choices on 8-s trials was increased to 4 food pellets in group 1-4, while the reward for correct choices on 2-s trials was increased to 4 food pellets in group 4-1. The reward remained at a single food pellet for the lever associated with the opposing duration in both groups. All other aspects were the same as the baseline testing phase; there were 15 test sessions, with retraining sessions intervening between 5-session blocks of testing.

Data analysis

Psychophysical functions were produced by plotting the proportion of long lever responses as a function of signal duration for each group. Temporal discrimination parameters were then calculated by fitting a linear function to the middle five values of the psychophysical function in Experiments 1 and 2 and to the middle seven values of the psychophysical function in Experiment 3, modelling after Church and Deluty (1977). This method has been commonly employed in the literature to characterize psychophysical functions for time (e.g., Droit-Volet, Clément & Fayol, 2008; Wearden, 1995). From the linear function, a point of subjective equality (PSE) index was computed by determining the duration associated with 50% long responses. The PSE gives an index of the point where the individual was indifferent between the two anchor durations. A difference limen (DL) measure was also computed as an index of the accuracy of temporal discrimination. The DL was equal to the duration associated with 75% long response minus the duration associated with 25% long response divided by 2.

All rats successfully achieved and maintained criterion during training, baseline and reward manipulation sessions. All 15 sessions for both the baseline and reward magnitude manipulation phases were included in the analysis.

Results

Psychophysical functions

Figure 1 shows the psychophysical functions relating the proportion of “long” responses and signal duration in groups 1-4 (top panel) and 4-1 (bottom panel) during the baseline (1-1) and reward magnitude manipulation (1-4 or 4-1) phases. The psychophysical functions were flatter during the reward magnitude manipulation. In group 1-4, the flattening of the function is apparent on both sides, but is more pronounced on the short side of the function. On the other hand, in group 4-1, the function is flattened primarily on the long side. The psychophysical functions were analyzed separately for each group by an ANOVA with the variables of Duration and Phase. For group 1-4, this revealed a significant effect of Duration, F(8,40) = 184.1, p < .001, and Phase x Duration, F(8,40) = 7.3, p < .001, but no effect of Phase, F(1,5) = 1.0. Tukey post-hoc tests compared the response during the two phases at each duration in a pair-wise manner and indicated that there were significant differences at durations 2.00, 2.38, 2.83, 3.36, and 4.00 s. A similar pattern of results was found in group 4-1, with significant effects of Duration, F(8,40) = 152.1, p < .001 and Phase x Duration, F(8,40) = 2.7, p < .05, but not Phase, F(1,5) = 1.7. Tukey post-hoc tests indicated that there were significant differences at durations 5.66, 6.73, and 8.00 s.

Figure 1.

Psychophysical functions plotting proportion of ‘long’ responses against test duration during the baseline (1-1) and reward magnitude manipulation (1-4 or 4-1) phases for groups 1-4 (top panel) and 4-1 (bottom panel) in Experiment 1.

Temporal discrimination parameters

The PSE and DL values are presented in Table 1 for each rat, as well as the mean and standard error of the mean (SEM) for each group during the baseline (1-1) and reward magnitude manipulation (1-4 or 4-1) phases. There was a decrease in PSE for all 6 rats in group 1-4 and an increase in PSE in 5 of 6 rats in group 4-1. An ANOVA conducted on the PSE values for each group with the variable of Phase indicated a significant decrease in the PSE in group 1-4, F(1,5) = 67.0, p < .001, but no significant change in group 4-1, F(1,5) = 1.8. There was also a large effect of the reward magnitude manipulation on the DL values, with all 6 rats in group 1-4 and 5 of the 6 rats in group 4-1 displaying an increase in DL. The effect on the DL was larger in group 1-4 (1.02 vs. 1.75) than in group 4-1 (0.97 vs. 1.41), but the ANOVAs revealed significant increases in DL in both groups: 1-4, F(1,5) = 10.8, p < .05 and 4- 1, F(1,5) = 7.1, p < .05.

Table 1.

Point of subjective equality (PSE) and difference limen (DL) values for individual rats in groups 1-4 and 4-1 during the baseline (1-1) and reward magnitude manipulation phases (1-4 or 4-1) of Experiment 1.

Group 1-4	PSE		DL		Group 4-1	PSE		DL
Rat	1-1	1-4	1-1	1-4	Rat	1-1	4-1	1-1	4-1
1	4.39	4.00	0.93	1.54	1	4.51	5.22	1.03	1.53
2	3.77	3.53	0.94	2.23	2	4.62	4.74	0.84	1.01
3	4.35	4.04	0.97	1.03	3	3.79	5.32	0.94	1.95
4	4.48	4.29	1.11	1.56	4	4.70	4.06	1.35	1.21
5	4.48	4.08	1.11	1.59	5	4.34	5.10	0.82	1.54
6	3.50	3.27	1.06	2.53	6	4.34	4.35	0.82	1.20
Mean	4.16	3.87	1.02	1.75	Mean	4.38	4.80	0.97	1.41
SEM	0.17	0.16	0.03	0.22	SEM	0.13	0.21	0.08	0.14

Discussion

The increase in reward magnitude for correct choices on 2- or 8-s trials caused a flattening of the psychophysical function (Figure 1) evidenced by a substantial increase in DL (Table 1). The effect was asymmetrical in that the primary differences occurred at the durations on the opposite side of the PSE from the duration that was associated with the larger magnitude. The asymmetrical nature of the reward magnitude manipulation was particularly notable in group 1-4 where there was an additional effect on the PSE. The decrease in accurate responding on the side of the function associated with the smaller reward may be due to a contrast effect towards the response with the increased reward magnitude. However, there was no evidence for an increase in accuracy on the side of the function associated with the larger magnitude. If anything, the trend was towards a reduction in accuracy, and therefore, reduced preference for longer durations to the right of the PSE.

An increase in pacemaker speed in SET (Gibbon & Church, 1984) should have resulted in a decrease in the PSE in both groups, but this was not the case in the present experiment. While group 1-4 produced a significant decrease in the PSE, group 4-1 demonstrated no significant change in PSE (but a trend towards an increased PSE). So, this result is only partially consistent with predictions of an arousal-based increase in clock speed, according to SET. However, an increase in the SET pacemaker rate would not have increased the DL, which was the most robust effect in the present experiment, with both groups showing a flattening of the psychophysical function. It is therefore unlikely that a change in pacemaker speed alone can account for the results of the present experiment (although this will be investigated further in Experiment 3).

Alternatively, BeT (Killeen & Fetterman, 1988) predicts that the increase in magnitude should have increased pacemaker rate and this would lead to a sharpening of the psychophysical function. According to this account, the DL should have decreased during the reward magnitude manipulation phase. In contrast, the DL significantly increased when the reward magnitude for one of the choices increased, so this result is counter to the predictions of BeT.

In addition, the BEM model (Jozefowiez et al., 2009) predicts that the increase in magnitude should have induced a bias to choose either the long or short sample only, without any consequential effects on the DL, so this model would also appear to fall short in predicting the present results.

The final possibility is that the reward magnitude manipulation might have affected attention to the task by altering the opening/closure latency of the switch and/or switch fluctuation rate in SET. Based on previous research (Galtress & Kirkpatrick, 2009), one might expect that increasing reward magnitude would increase attention, thereby sharpening the psychophysical function. This was not found here. Instead, it seems that increasing the reward magnitude produced a general disruption in attention to the task or in stimulus control. This result is similar to the findings of Ward and Odum (2006, 2007) in that they reported a decrease in the slope of the psychophysical function when they gave a pre-feeding manipulation. However, the present study found a similar disruption effect under increases in reward magnitude as opposed to decreases in reward value. This pattern is somewhat surprising given the previous results in the literature with the peak and DRL procedures indicating opposing effects of reward increases and decreases on the peak (Doughty & Richards, 2002; Galtress & Kirkpatrick, 2009; Grace & Nevin, 2000; Kacelnik & Brunner, 2002; Ludvig et al., 2007; Roberts, 1981) when a single temporal duration was investigated. On the other hand, non-systematic disruptions in temporal discrimination have been found with dopaminergic drugs (Cevik, 2003; Chiang et al., 2000; Lejeune et al., 1995; Meck, 1996; Santi, Weise & Kuiper, 1995; Tofighy et al., 2003), which are normally purported to produce systematic effects on peak procedure timing (Buhusi & Meck, 2002; Cevik, 2003; Maricq & Church, 1983; Maricq et al., 1981; Meck, 1983, 1986, 1996). Thus, finding a general disruption in temporal discrimination may not be unexpected in the present experiment.

This overall reduction in accuracy due to differences in reward magnitude is also contrary to what would be expected as a result of a differential outcomes effect (Carlson & Weilkiewicz, 1976; Urcuioli, 2005), as performance should be facilitated by discriminative outcomes. However, to our knowledge, differential outcomes have not been examined previously in temporal discrimination using different reward magnitudes. It appears that under these circumstances, instead of promoting temporal discrimination, differential outcomes impair discrimination. This could be because the bisection task may be evoking reward value computation as in temporal discounting tasks (e.g., Mazur, 2001).

Temporal discrimination is a distinct task compared to those usually found in differential outcomes procedures. Here, sustained focus to the task is necessary for accurate temporal judgement, whereas the usual discriminative stimuli in the differential outcomes require judgements of sensory perception (e.g., wavelength). The interaction of reward magnitude/value and delay that has been demonstrated repeatedly may disrupt the discrimination of two temporal intervals with different outcomes in a way that would not be a factor in discrimination between two perceptual events that are not affected by the reward value/delay interaction.

Both BeT and SET (Gibbon & Church, 1984; Killeen & Fetterman, 1988) predict that it is the contrast of the pacemaker speed in the baseline training phase and the testing phase that would produce the shifts in the function. In other words, the reference memory must have been established with an initial pacemaker rate to see an effect of a change in pacemaker rate on responding. Thus, these timing models would predict no effect of the pacemaker process on discrimination learning if the reward magnitude difference was present from the beginning of training. On the other hand, if the difference in magnitude produces a general deficit in attention to the task, then this effect might impair discrimination of the signal durations and thereby flatten the psychophysical function. Therefore, Experiment 2 delivered the reward magnitude manipulation from the beginning of temporal discrimination training to see whether the difference in reward magnitude was sufficient to cause a general disruption in performance without any previous training on alternative reward magnitudes.

Experiment 2

Timing models that include a pacemaker component propose that it is the contrasting effects of baseline training and subsequent testing under the reward magnitude shifts that result in transient shifts in the expected time of reinforcement (Gibbon & Church, 1984; Killeen & Fetterman, 1988). However, Galtress and Kirkpatrick (2009) and Ludvig et al. (2007) both reported longer-lasting shifts in timing on a peak procedure, suggesting that changes in clock speed may not have been responsible for the reward magnitude/value change effects on timing. One easy means of differentiating between clock and attentional explanations is to train differences in reward magnitude within the context of a temporal discrimination procedure from the beginning. In this case, the clock speed would be the same throughout training and there would be no basis for any effect of differential reward magnitudes on the psychophysical function. On the other hand, attentional effects on timing would be expected to impact on performance as poor attention to the task would impede discrimination of the different signal durations.

Two groups of rats received 1 food pellet for correctly responding to either a 2- or 8-s houselight signal, while receiving 4 food pellets for correctly responding to the alternative signal. These two groups were contrasted with two control groups that received either 1 or 4 food pellet(s), for correctly responding to both signal durations.

Method

Animals

Twenty-four male hooded Lister rats (Charles River, UK) were approximately 12 weeks old with a mean ad libitum weight of 360 g (range = 300 – 385 g). All aspects of housing and husbandry were the same as in Experiment 1.

Apparatus

The apparatus was the same as that used in Experiment 1.

Procedure

Rats were randomly allocated to one of four groups: 1-1, 1-4, 4-1 and 4-4 (n = 6), where the group labels refer to the reward magnitude for correct choices on 2- and 8-s trials, respectively.

Pre-training

Pre-training was carried out over 3 days. On day 1, all rats received magazine training with food delivery on a VT 60-s schedule for 60 min. Group 1-1 received a single food pellet and group 4-4 received 4 food pellets throughout, and groups 1-4 and 4-1 both received 1 or 4 food pellets with a probability of 0.5. The following 2 days consisted of CRF training, with food delivered for a single lever press on both the left and right levers, one per day counterbalanced, for a total of 30 lever presses. The reward magnitude for left and right lever responses was matched with the magnitude that would later be received in training, either 1 or 4 pellets.

Training

The training phase was the same as in Experiment 1. However, in group 1-1, correct choices to both durations resulted in a single food pellet reward, in group 1-4 the lever associated with the 8-s duration delivered 4 food pellets, as did the lever associated with the 2-s duration in group 4-1, while the lever associated with the opposing duration delivered a single food pellet in both groups. Group 4-4 received 4 food pellets for correct responses to both signal durations. Rats failing to achieve at least 80% correct after 10 sessions were given a remedial procedure. At the onset of each session an additional 14 (7 per training duration) forced choice trials were presented. Here, one lever only was inserted and the corresponding reward was delivered after a response on that lever; the lever was then withdrawn and the ITI initiated. This procedure was maintained until a criterion of 90% correct responses during the remaining discrimination trials over three consecutive sessions was reached. The rats were then returned to the original training procedure minus the forced choice trials and were again required to reach criterion prior to being tested.

Testing

The test sessions were the same as the baseline testing phase in Experiment 1. The reward magnitude on training trials was the same as in the training phase.

Data analysis

Data analyses were conducted as in Experiment 1. None of the rats in group 4-4 achieved the required criterion of 90% correct responding over three consecutive non-remedial sessions during training and so were excluded from the experiment prior to the test phase. All other rats successfully achieved and maintained criterion during training and test sessions. All 15 test sessions were included in the analysis.

Results

Acquisition

Both group 1-4 and group 4-1 took significantly longer than group 1-1 to reach criterion (group 1-4, 62 sessions; group 4-1, 55 sessions; group 1-1, 21 sessions), F(2,17) = 8.6, p < .01. Tukey post-hoc tests revealed significant differences between group 1-4 and 1-1 and group 4-1 and 1-1. There was no difference in acquisition between groups 1-4 and 4-1.

Psychophysical functions

Figure 2 shows the psychophysical functions for group 1-4 (top panel) and group 4-1 (bottom panel) compared to group 1-1 in both cases. The psychophysical functions for groups 1-4 and 4-1 both appear flatter than the psychophysical function for group 1-1, on both sides of the function. The flattening effect seems slightly more pronounced in group 1-4 on the short side of the function. An ANOVA compared groups 1-1 and 1-4 with the variables of Duration and Group; this revealed significant effects of Duration, F(8,80) = 175.3, p < .001, and Duration x Group, F(8,80) = 3.5, p < .01, but no main effect of Group, F(1,10) < 1. Tukey follow-up tests on the Duration x Group interaction comparing the two groups at each duration indicated significant group differences at the following durations: 2.83, 3.36, and 5.66 s. A further ANOVA comparing groups 4-1 and 1-1 disclosed a significant effect of Duration, F(8,80) = 233.0, p < .001. There was a near-significant interaction, F(8,80) = 2.0, p = .06, but no main effect of Group, F(1,10) < 1.

Figure 2.

Psychophysical functions plotting proportion of ‘long’ responses against test duration for group 1-1 compared to group 1-4 (top panel) and group 4-1 (bottom panel) in Experiment 2.

Temporal discrimination parameters

The temporal discrimination parameters (PSE and DL) are displayed in Table 2. The PSE was near the geometric mean of 4 in all three groups (1-1: PSE = 4.26, 1-4: PSE = 4.06, and 4-1: PSE = 4.10). There were no differences in the PSE in group 1-4 and group 4-1 when compared to group 1-1, both Fs(1,10) < 1. On the other hand, the DL was larger in groups 1-4 (1.51) and 4-1 (1.21) compared to group 1-1 (0.92) due to a flattening of the psychophysical function induced by the reward magnitude manipulation. Statistical analyses confirmed an increase in the DL for group 1-4, F(1,10) = 8.2, p < .05, and also for group 4-1, F(1,10) = 5.9, p < .05, compared to group 1-1.

Table 2.

Point of subjective equality (PSE) and difference limen (DL) values comparing individual rats in groups 1-1, 1-4, and 4-1 in Experiment 2. The data in different columns are from individual rats in each of the different groups.

Rat	1-1	PSE 1-4	4-1	1-1	DL 1-4	4-1
1	4.18	3.75	4.62	1.02	1.39	1.17
2	4.50	4.21	4.40	0.78	1.72	0.95
3	4.22	3.25	2.38	0.87	2.34	1.64
4	3.80	5.05	4.70	0.80	1.52	1.19
5	4.17	4.07	4.48	0.91	0.94	1.36
6	4.67	4.00	3.99	1.14	1.17	0.96
Mean	4.26	4.06	4.10	0.92	1.51	1.21
SEM	0.11	0.22	0.33	0.06	0.20	0.11

Discussion

The above results suggest that prior training on the baseline (1-1) condition was not necessary to disrupt performance under the reward magnitude manipulation. The DL values in both groups 1-4 and 4-1 were clearly larger than the DL values in group 1-1, but there was no evidence of a difference in PSE among the groups. This suggests that the shift in the psychophysical function (PSE) in Experiment 1 may have been due to a pacemaker effect, while the decrease in slope may have reflected a separate attention-based process.

The results of Experiment 2 concur with Experiment 1 in that they are contrary to expectations of the differential outcomes effect (Carlson & Weilkiewicz, 1976; Urcuioli, 2005). The differential outcomes effect would predict that the different reward magnitude for each of the two durations in group 1-4 and group 4-1, compared to group 1-1, would enhance discrimination between the two intervals and facilitate acquisition in these groups. That this was not the case, however, lends further support to the notion that overall discrimination was disrupted in the present task due to the difference in and interaction of both delay and amount.

The present experiment also highlighted concerns about an additional effect of the reward magnitude manipulation. Specifically, the lack of successful discrimination learning in group 4-4 may have been due to satiety from the large number of food pellets delivered in each session. Satiety effects could possibly have occurred in groups 1-4 and 4-1 as well, albeit to a lesser degree. If so, the satiety effects could be at least partially responsible for the effects of the reward magnitude manipulations on the psychophysical function. Satiety, induced through pre-feeding, produces a general disruptive effect on temporal discrimination tasks (Ward & Odum, 2006, 2007), so it is important to control for any satiety effects on performance. There are two main approaches to dealing with satiety issues: (1) by equating the amount of reinforcement given during any phase and (2) by delivering more sparsely-spaced reinforcers. Experiment 3 will capitalize on both of these approaches.

The lack of a difference in the PSE between either of the experimental groups and group 1-1 in the present experiment may be indicative of the role of pacemaker effects in Experiment 1, as there was a shift in the PSE in group 1-4 following an increase in reward magnitude for correct long responses. Therefore, Experiment 3 will address the nature of the effects on the PSE found with group 1-4 in Experiment 1.

Experiment 3

Experiment 1 produced a flattening of the psychophysical function in both group 1-4 and group 4-1. This was coupled with a shift in the PSE in group 1-4 only. The present experiment aimed to further investigate this effect with a similar group receiving baseline training of 1 food pellet for 2- and 8-s durations, then a shift in reward magnitude to 4 pellets for the 8-s duration. However, in this experiment, the session length was increased and the amount of reinforcement was more closely matched over each phase. This was to lessen any satiety effects that may have produced a general disruption in performance. An additional group was given the same procedure, but with durations of 4 and 16 s. Increasing both signal durations proportionally while keeping the reward magnitude consistent at 1 and 4 pellets reduces the likelihood of bias from the larger magnitude reward as increasing delay reduces the disparity in reward value (through delay discounting; e.g., Mazur, 1987). Additional test durations were also included that ranged outside of the signal durations. This was to further establish if any systematic shifts in the location of the psychophysical function could be found.

Method

Animals

Twelve male hooded Lister rats (Charles River, UK) were approximately 14 weeks old with a mean ad libitum weight of 376 g (range = 350 – 410 g). The rats were housed in a vivarium that was maintained on a 12:12 hr light-dark cycle, with light onset at 8 am. All aspects of housing and husbandry were conducted as in Experiment 1.

Apparatus

The apparatus was the same as in Experiment 1.

Procedure

The rats were randomly allocated to one of two groups (n = 6): group Short with signal durations of 2 and 8 s and group Long with signal durations of 4 and 16 s.

Pre-training

Pre-training was carried out in 14-hr sessions that consisted of two blocks of trials, with an inter-block interval of 60-min. In the initial block, all rats received magazine training with a single food pellet delivered on a VT 60-s schedule for a total of 40 pellets. The following block consisted of CRF training, with a single food pellet reward delivered for a single lever press on both the left and right levers counterbalanced and presented individually for 15 presses each and then alternated, for a total of 30 lever presses each. The maximum session length was 14 hr. All rats completed pre-training in 2 sessions.

Training

Both groups were trained on a single food pellet reward for a correct response to both signal durations. The signal durations for group Short were 2 and 8 s; for group Long, 4 and 16 s. All training and correction trials were given as in Experiment 1. Each session consisted of four blocks with an inter-block interval of 60-min, with each block containing 44 training trials with 22 trials of each signal duration. All rats received 176 food pellets and a maximum session length of 14 hr. Training continued until each rat had reached a criterion of 90% correct for three consecutive sessions. No remedial training was necessary in this experiment.

Testing, baseline

Training progressed as the previous phase, except here each sub-block now contained an additional 26 non-reinforced test trials; two of each duration. Group Short received test trials of 1.19, 1.41, 1.68, 2.38, 2.83, 3.36, 4.00, 4.76, 5.66, 6.73, 9.51, 11.31 and 13.45 s, and group Long received test trials of 2.38, 2.83, 3.36, 4.76, 5.66, 6.73, 8.00, 9.51, 11.31, 13.45, 19.03, 22.63 and 26.91 s. The baseline testing phase consisted of 9 sessions with a single retraining session after every 3 sessions. A total of 176 pellets were delivered over the course of a 14-hr session.

Testing, reward magnitude manipulation

In the reward magnitude manipulation phase, the reward for a correct choice following the long duration (8 s for group Short or 16 s for group Long) was increased to 4 food pellets. The reward remained at a single food pellet for the lever associated with the opposing duration in both groups. All test trials were the same duration as baseline. However, each block consisted of 35 trials; 11 trials of each of the two signal durations, and 13 test trials, 1 of each non-reinforced duration. Eighteen sessions were given, with 2 retraining sessions after each set of 6 sessions. This equated the total test trials given in the baseline and reward magnitude manipulation phases. Each animal received 220 food pellets per session with a maximum session length of 14 hr.

Data analysis

Data analyses were conducted as in Experiment 1. All rats in group Short met criterion during training and all successive phases. One rat in group Long did not achieve criterion during baseline testing and so was excluded prior to the reward magnitude phase and the data removed from the analysis. All 9 baseline testing and 18 reward magnitude manipulation sessions were included in the analysis of the remaining rats.

Results

Psychophysical functions

Figure 3 shows the psychophysical functions during the baseline and reward magnitude manipulation phases for group Short (top panel) and group Long (bottom panel). In both groups, the reward magnitude manipulation flattened the psychophysical function on both sides compared to baseline, but the effect was larger on the short side of the function. Separate ANOVAs were conducted on each group’s psychophysical functions with the variables of Phase and Duration. For group Short, this revealed effects of Phase, F(1,5) = 8.8, p < .05, Duration, F(14,70) = 462.8, p < .001, and Phase x Duration, F(14,70) = 8.2, p < .001. Tukey post-hoc tests on the interaction revealed significant differences at durations 2.83, 3.36, and 13.45 s. For group Long, the effects of Duration, F(14,56) = 896.8, p < .001, and Phase x Duration, F(14,56) = 4.0, p < .001, were significant, but the effect of Phase, F(1,4) = 5.2, did not reach statistical significance (p = .08). Tukey post-hoc tests on the interaction revealed differences at durations 4.76, 5.66, 6.73, and 11.31 s.

Figure 3.

Psychophysical functions plotting proportion of ‘long’ responses against test duration during the baseline (1-1) and reward magnitude manipulation (1-4) phases for groups Short (top panel) and Long (bottom panel) in Experiment 3.

Temporal discrimination parameters

The temporal discrimination parameters of PSE and DL are presented in Table 3. The PSE decreased in all 6 rats in group Short during the reward magnitude manipulation (1-4), and this effect was also seen in 4 of the 5 rats in group Long. An ANOVA with the variable of Phase was conducted on each group’s PSE data and this disclosed a significant decrease in the PSE in group Short, F(1,5) = 13.3, p < .05. There was a trend towards a significant decrease in group Long, F(1,4) = 4.4, p = .1, but this was not statistically verified. In examining the DL values, all rats in both groups displayed an increase in the DL. This trend was verified by an ANOVA: group Short, F(1,5) = 7.4, p < .05, and group Long, F(1,4) = 18.5, p < .05.

Table 3.

Point of subjective equality (PSE) and difference limen (DL) values for individual rats in groups Short and Long during the baseline (1-1) and reward magnitude manipulation phases (1-4) of Experiment 3.

Group Short	PSE		DL		Group Long	PSE		DL
Rat	1-1	1-4	1-1	1-4	Rat	1-1	1-4	1-1	1-4
1	4.27	4.08	0.91	0.95	1	8.07	8.00	1.74	2.45
2	4.01	3.58	0.86	1.20	2	7.99	8.03	2.07	2.69
3	4.24	3.89	0.85	1.06	3	8.53	7.34	1.73	3.13
4	3.83	3.00	0.86	2.09	4	8.40	8.03	1.81	2.42
5	3.38	3.28	1.18	1.54	5	8.67	7.71	2.12	2.50
6	3.77	2.44	0.99	1.89
Mean	3.99	3.38	0.94	1.46	Mean	8.33	7.82	1.89	2.64
SEM	0.09	0.25	0.05	0.19	SEM	0.13	0.13	0.08	0.13

Discussion

The present experiment replicated the effects on the DL found in Experiment 1. Both groups displayed a flattening of the psychophysical function, indicative of a general disruption in performance that was unaffected by the training durations. The present experiment aimed to diminish any impact of satiety on the DL by increasing session lengths to induce a leaner rate of food-earning and increase the similarity in food-earning rates in the baseline (0.6 g/hr) and reward magnitude (0.7 g/hr) manipulation phases. The results indicated that the effects of the reward magnitude manipulation were still robust even though satiety differences were greatly reduced. This indicates against any significant contribution of satiety to the reward magnitude manipulation.

Consistent with the outcome of Experiment 1, group Short produced a decrease in the PSE, which is consistent with an increase in pacemaker rate (e.g., Meck, 1996). However, this effect was greatly reduced in group Long, when the signal durations were increased to 4 and 16 s. Therefore the effects of the reward magnitude increase in group Short (and, presumably also group 1-4 in Experiment 1, which was identical to group Short in this experiment) may have been due to an increased preference for the lever associated with the larger magnitude. Increasing the signal durations in group Long should have reduced any lever bias because the overall value of the long-signal reward would be decreased due to temporal discounting (e.g., Mazur, 1987). This was indeed the case.

General Discussion

Increasing reward magnitude for correct responses to the short (Experiment 1) or long signal (Experiments 1 and 3) altered the psychophysical function; this effect was also apparent when differential reward magnitudes were presented throughout training (Experiment 2). The most robust effect across the three experiments was a flattening of the psychophysical function, measured by an increase in the DL. When the reward magnitude was shifted between phases in Experiments 1 and 3, the effects on the psychophysical function were asymmetrical in a subset of conditions, resulting in a decrease in PSE in group 1-4 in Experiment 1 and in group Short (1-4 manipulation) in Experiment 3. These two different conditions were identical – there was an increase in reward magnitude from 1 to 4 pellets for responses associated with an 8-s signal in a 2 vs. 8-s discrimination, and they were the only conditions in which there was a significant effect on the PSE. On the other hand, in Experiment 2, where the reward magnitude difference was present throughout training, the effects were more symmetrical in nature.

The results are consistent with two separable effects of reinforcer magnitude on performance: a within-session contrast effect due to the presence of different reward magnitudes for correct responses to the short vs. long signals (which was apparent in all three experiments), and a between-session contrast effect due to the change in reward magnitude between the baseline and reward magnitude manipulation phases. The within-session contrast produced increases in the DL, whereas the between-session contrast produced a decrease in the PSE when the magnitude increase was associated with the 8-s signal. When the signal durations were increased to 4 and 16 s (group Long, Experiment 3), the effect of the magnitude increase on the PSE was mitigated, which may have been due to a decrease in the subjective value of the larger reward due to temporal discounting (Mazur, 2001; Rachlin, 2000).

The flattening of the psychophysical function due to within-session reward contrast did not appear to be attributable to any satiety effects on performance, as demonstrated in Experiment 3, thereby arguing against the BeT pacemaker account advanced by Ward and Odum (2006). Specifically, they argued that satiety effects due to pre-feeding prior to psychophysical testing led to a decrease in the speed of the BeT pacemaker, which reduced discriminability of the durations. The fact that the increase in the DL was still apparent when satiety effects were greatly reduced indicates against this account.

The effects of the reward magnitude manipulations on the DL are consistent with a general disruption in stimulus control. Similar effects on temporal discrimination have been reported under response-independent food delivery (Ward & Odum, 2006; Wilkie, Symons & Tees, 1988), extinction (Killeen, Hall & Bizo, 1999; Ward & Odum, 2006, 2007), presentation of distracting stimuli (Sutton & Roberts, 2002; Ward & Odum, 2007), and pharmacological manipulations such as the administration of dopamine agonists (Cevik, 2003; Chiang et al., 2000; Lejeune et al., 1995; McClure, Saulsgiver & Wynne, 2005; Santi et al., 1995; Tofighy et al., 2003) and morphine (Huang, Edwards, Rounis, Bhatia & Rothwell, 2005; Ward & Odum, 2005).

Previous studies have suggested that the decreases in stimulus control accompanying pharmacological and behavioral manipulations may be due to a decrease in attention to the relevant stimulus dimension (Blough, 1996; Heinemann, Avin, Sullivan & Chase, 1969; Santi et al., 1995), in this case time. The role of attention was supported in Experiment 2 when the flattening of the psychophysical function was reported without any changes in reward magnitude. Both SET and BeT predict that this manipulation should have produced no effect if the mechanism of action operated on pacemaker speed (Gibbon & Church, 1984; Killeen & Fetterman, 1988). In addition, Galtress and Kirkpatrick (2009) concluded that the effects of between-session contrast on peak procedure responding were most likely due to attentional effects on the switch due to two factors: (1) the effects of reward magnitude shifts on timing were persistent, which is inconsistent with a clock speed account; and (2) the effects of reward manipulations on different FI durations indicated both an additive (e.g., switch latency) and multiplicative (e.g., switch fluctuation) component. In the present study, a fluctuating switch would add noise to timing of the durations and this would in turn increase the DL. Switch latency effects would operate to shift the psychophysical function, thereby altering the PSE. It is possible that both of these effects were operating here as well. However, the effect of reward magnitude shifts on the PSE in Experiments 1 and 3 is also consistent with the induction of a bias for the lever associated with the larger reward, an idea which is supported by the diminished change in the PSE when the signal durations were increased in Experiment 3 (group Long). Therefore, it is difficult to differentiate between the contributions of attention and bias to the effects on the PSE, but the results seem to point preferentially towards a response bias effect.

While general disruptions in temporal discrimination have been reported in several studies, there are also a number of reports indicating systematic disruptions to the psychophysical function in the form of changes in the PSE as was observed in Experiments 1 and 3. Specifically, shifts in the PSE have been found with both extinction and pre-feeding manipulations (McClure et al., 2009), and also the administration of dopaminergic agonists and antagonists (Buhusi & Meck, 2002; Cevik, 2003; Maricq & Church, 1983; Maricq et al., 1981; Meck, 1983, 1986, 1996). However, Meck (1996) also produced evidence of a general disruption in performance, with three-quarters of the rats in the study showing an additional flattening of the psychophysical function. These results have been interpreted as due to clock speed effects on performance. The present studies clearly argue against changes in clock speed for a number of reasons. Changes in clock speed would not have affected the DL, so this cannot be the sole explanation for the results of the present experiments. And, according to BeT, a clock speed effect should have decreased the DL in Experiments 1 and 3, whereas increases in the DL were found instead. Most importantly, however, are the results of Experiment 2 demonstrating increases in the DL in the absence of any between-session shifts in magnitude. Neither of these timing models would predict any effects in this instance through a clock speed mechanism.

In summary, the present experiment lends further support to Galtress and Kirkpatrick’s (2009) assertion that the reward magnitude effects operated on attention to time. However, attention is poorly understood both empirically and as a psychological construct in the context of timing models, and it therefore seems necessary to continue to engage in further explorations to illuminate the role of attention in timing processes, how various disruptors might alter attention to time, and to disentangle different aspects of attention in the timing process. For example, why would an increase in reward magnitude associated with one of the signal durations disrupt attention in temporal discrimination when it appears to instead promote attention in the peak procedure? An additional important contribution of the present paper is that it adds to the growing consensus that timing and motivation are not wholly independent. Although the recently devised BEM model (Jozefowiez et al., 2009) did not predict the present results, this model represents an initial promising step forwards in integrating timing and reward processing. The results of the present series of studies should prove informative for aiding the growth and development of the next generation of computational models in the field as well as in furthering our understanding of the processes involved in anticipation of upcoming events and the allocation of behavior that accompanies those expectations.