Effects of novelty and methamphetamine on conditioned and sensory reinforcement

Abstract

Background

Light onset can be both a sensory reinforcer (SR) with intrinsic reinforcing properties, and a conditioned reinforcer (CR) which predicts a biologically important reinforcer. Stimulant drugs, such as methamphetamine (METH), may increase the reinforcing effectiveness of CRs by enhancing the predictive properties of the CR. In contrast, METH-induced increases in the reinforcing effectiveness of SRs, are mediated by the immediate sensory consequences of the light.

Methods

The effects of novelty (on SRs) and METH (on both CRs and SRs) were tested. Experiment 1: Rats were pre-exposed to 5 s light and water pairings presented according to a variable-time (VT) 2 min schedule or unpaired water and light presented according to independent, concurrent VT 2 min schedules. Experiment 2: Rats were pre-exposed to 5 s light presented according to a VT 2 min schedule, or no stimuli. In both experiments, the pre-exposure phase was followed by a test phase in which 5 s light onset was made response-contingent on a variable-interval (VI) 2 min schedule and the effects of METH (0.5 mg/kg) were determined.

Results

Novel light onset was a more effective reinforcer than familiar light onset. METH increased the absolute rate of responding without increasing the relative frequency of responding for both CRs and SRs.

Conclusion

Novelty plays a role in determining the reinforcing effectiveness of SRs. The results are consistent with the interpretation that METH-induced increases in reinforcer effectiveness of CRs and SRs may be mediated by immediate sensory consequences, rather than prediction.

Introduction

Stimuli such as light onset, which do not have obvious important biological effects such as maintaining homeostasis and/or reproduction (e.g., food, water, and sex), are often considered to be neutral and to acquire motivational effects only through pairing with other biologically important reinforcers. For this reason, initial reports that light onset, which does not have important biological effects, could be a primary reinforcer generated interest [1–4]. The term sensory reinforcer (SR) has been used to refer to sensory stimuli, such as light onset, which were found to have primary reinforcing effects [For reviews see, 5, 6–9]. Investigations into the reinforcing effects of light have reported that both light onset and offset act as SRs, although light onset is a stronger reinforcer than light offset [3, 9–13] . Additionally, studies have reported that incrementing or decrementing a light from an intermediate value can be reinforcing [14–17].

Psychomotor stimulants such as amphetamine and nicotine have been reported to increase responding for a light onset SR [18–24]. Caggiula and colleagues [25–33] have suggested that the effects of nicotine on the reinforcing effects of SRs may play an important role in nicotine self-administration.

Light onset is often paired with biologically important primary reinforcers through Pavlovian conditioning. Following conditioning, light onset is considered to be a conditioned stimulus (CS). There is also strong evidence that responding for these light stimuli, which are considered to be conditioned reinforcers (CRs), is increased by psychomotor stimulants [34–41].

Winterbauer and Balleine [22] compared the effects of amphetamine on the onset of a light that was a SR and the same light as a CR following Pavlovian pairing with sucrose solution. Consistent with previous reports, they found that amphetamine increased responding for light as both a SR and CR. These authors point out that explanations of the rate increasing effects of psychomotor stimulants on responding for CRs often emphasize the prediction of biologically important reinforcers. Such explanations theorize that this prediction signal involves dopamine (DA) neurotransmission [42–44]. The implication is that enhancement of the DA mediated predictive properties of the CR by psychomotor stimulants increases the response enhancing properties of the CR. In contrast, no such explanation can be given for psychomotor stimulant-induced increases in responding for light onset when it is a SR. Increases in responding for a SR after administration of a psychomotor stimulant are mediated by the immediate sensory consequences of the light rather than the prediction of a future reinforcer.

Dopamine and Sensory Reinforcement

An alternative account of the relationship between DA and sensory stimuli that emphasizes the immediate sensory aspects of a stimulus rather than its predictive properties is provided by Redgrave and colleagues [45]. According to these authors, novel sensory stimulation results in the phasic firing of DA neurons. The resulting release of DA in the striatum increases the probability that the animal will repeat the response that produced the novel sensory stimulation. Through this DA mediated mechanism, the organism learns about contingencies between its actions and sensory events. These events may or may not have immediate important biological consequences. They argue that the phasic DA responses, evoked by novel sensory stimuli, provide a signal independent of normal goal-directed reward systems (food, drink, temperature, sex, etc.). This SR system reinforces acquisition of basic action-outcome associations that are necessary for the development of complex novel goal-directed behavior.

The hypothesis of DA mediation of SRs has important implications for the addictive properties of methamphetamine (METH) and other psychomotor stimulants. While the reinforcing effectiveness of SRs may be weak in comparison to biologically important reinforcers, SRs are ubiquitous in the natural environment. Because they are so widespread, the role of SRs in guiding behavior may be underappreciated. According to Redgrave’s theory, enhancement of the reinforcing effectiveness of SRs by psychomotor stimulants would have the effect of making a wider variety of stimuli reinforcing, thereby increasing the control that these stimuli exert over behavior [46, 47].

Previously, we have reported that onset of a light for 5 s in an otherwise dark experimental chamber is a SR [48–50]. In the present experiment, Pavlovian conditioning is used to pair the same light stimulus with water, potentially making the light a conditioned stimulus. After Pavlovian conditioning of the CS with water, we compare responding for light onset as a CR to responding for light onset as a SR in rats that did not receive Pavlovian pairing of light onset with water.

Redgrave’s hypothesis (described above) predicts that decreasing the novelty of the light through pre-exposure should decrease its effectiveness as a SR. To test this prediction, we compare responding for the SR between rats that have been pre-exposed to the light and rats that have not been pre-exposed. Novelty is manipulated by pre-exposing subjects to a previously novel stimulus. In conditioning experiments the effect of decreasing novelty by pre-exposure to the CS is often called "the pre-exposure effect". Because Pavlovian conditioning involves pre-exposure to pairings of light and water, we also tested the effects of pre-exposure to random unpaired presentations of light and water. We predicted that random unpaired presentation of light and water would also decrease the effectiveness of light as a SR.

Psychomotor stimulants have been reported to increase responding for both SRs and CRs. We tested the effects of METH on responding for the light as a SR and as a CR. Of primary interest was the determination of differential effects of METH (if any) on responding for the light when it was a SR compared to responding for the light as a CR. If the effects of METH are dependent upon prediction of a biologically important reinforcer, then METH should have greater effects on responding for light onset as a CR. Conversely, if the effects of METH are mediated by the immediate sensory qualities of light onset, then METH should have similar effects on light onset as a SR and/or CR.

Materials and Methods

Subjects

One hundred and two male Holtzman Sprague Dawley rats that weighed between 320 and 470 g at the time of testing were used. Rats had ad libitum food. Water was restricted to 20 minutes of access per day at the conclusion of each day’s test session. Rats were housed in pairs in plastic cages (24 × 46 × 20 cm). The colony room lights turned on at 06:00 and off at 18:00. Rats were tested between 08:00 and 10:30. The study was approved by the Institution Animal Care and Use Committee of the State University of New York at Buffalo.

Apparatus

Sixteen in-house constructed experimental test chambers were used. Fig. 1 shows a schematic diagram of the experimental chambers and provides the precise dimensions of the test chamber. The back and two side walls of the test chambers were aluminum. The top and front of the chambers were made of Plexiglas. Flooring was made of parallel stainless steel rods at a distance of 1 cm. Each test chamber had three snout poke holes located in the left, right, and rear aluminum walls. Infrared photo detectors, placed 1 cm from the snout poke hole entrance were used to record the frequency and duration of snout pokes. Each snout poke hole provided access to an acrylic dish into which water could be pumped. Tygon tubing connected these acrylic dishes to 60 ml syringes mounted on Med Associates PHM-100 syringe pumps. Each test chamber was housed in a Coleman Cooler (Model # 3000000187), which blocked external audiovisual sources of stimulation. The syringe pumps were located externally to the sound attenuating coolers. An 800 MHz Pentium II computer connected to a Med Associates interface controlled the 16 chambers. The MED-PC IV software package was used to program and control experimental contingencies as well as collect data. The complete system operated at a temporal resolution of 0.01 s.

Fig. 1.

A schematic diagram of the experimental chambers. Note that the two Sonalerts and three lights on the side walls were not used in this experiment. Water was administered from the center rear hole during pre-exposure. See text for detailed description.

Experiment 1 Procedure

Drug Injections

(+) Methamphetamine (d-N, α-Dimethylphenethlyamine; d-Desoxyephedrine) hydrochloride was obtained from Sigma (Lot 054K0842). During the testing phase, rats received intraperitoneal injections of saline or METH dissolved in saline (0.5 mg/kg) 20 minutes prior to the start of each test session. Rats received a constant injection volume of 1.0 ml/kg. Doses were calculated as a salt.

Phase One Pre-exposure Sessions

The pre-exposure phase consisted of ten 30 minute sessions. Rats were assigned to one of two pre-exposure conditions which involved non-contingent presentations of light and water. In the paired group, light always preceded water, while in the unpaired group there was no correlation between light and water presentations. All presentations of the water and/or the light occurred according to a variable-time (VT) 2 minute schedule. VT intervals were selected without replacement from a list of 20 values generated using a Fleshler-Hoffman progression [51]. Light onset consisted of illumination of the house light in the ceiling of the test chamber for 5 s. The house light produced an illuminance of 53 lux as measured from either the left or right snout poke holes. Except for these scheduled illuminations, the chamber was dark. Water presentation consisted of 100 µl water at the center snout poke hole.

Phase Two Testing Sessions

The test phase also consisted of ten 30 minute sessions. One of the two snout poke holes on the side walls of the experimental chamber was designated as the active hole and the opposite snout poke hole as the inactive hole. Snout pokes into the active hole resulted in a 5 s illumination of the house light according to a variable-interval (VI) 2 min schedule of reinforcement. The left or right snout poke hole that had fewer entries during pre-exposure (least preferred) was chosen to be the active snout poke hole during testing. Snout pokes into the inactive and center snout poke holes were recorded but had no programmed consequences. No water was presented during the testing phase. The conditions of the testing phase were identical for both groups. During the test phase, 14 rats in the paired condition and 12 rats in the unpaired condition were treated with saline, while 13 paired rats and 13 unpaired rats were treated with METH.

Experiment 2 Procedure

The second experiment examined responding for novel and familiar SRs. The experimental procedures were similar to those used in Experiment 1 except that different pre-exposure groups were used. In the novel SR condition, the “no light” (NoLt; n=25) group experienced neither water nor light during the 10 day pre-exposure phase. In the familiar SR condition, the “light” (LT; n=25) group experienced light onset presented according to a VT 2 min schedule during the pre-exposure phase. During the test phase, 13 rats in the NoLt condition and 12 rats in the LT condition were treated with saline, while 12 NoLt rats and 13 LT rats were treated with METH.

Data Analysis

Dependent Variables

The same dependent variables were used in both Experiments 1 and 2. Snout pokes to the active and inactive holes were the dependent measures. A snout poke was operationally defined as interruption of the infrared photobeam. Only one response was recorded for each photobeam interruption.

The distribution of active responding for animals in all four conditions was examined using Shapiro-Wilk tests. Tests conducted by group indicated non-normally distributed data for the paired, LT, and unpaired conditions [paired W(27) = 0.893, p = 0.009; LT W(25) = 0.859, p = 0.003; unpaired W(25) = 0.861, p = 0.003]. The NoLt group statistic was not significant but trended toward significance [W(25) = 0.929, p = 0.081]. The log₁₀ transform of the frequency of snout pokes was found to normalize the variances and to produce non-significant Shapiro-Wilk statistics for all conditions. This transformation was used in statistical tests of the absolute active and inactive response rates.

The relative frequency of active responding (RFActive) was also used as a dependent measure. The RFActive measure allows preference to be measured independently of the absolute frequency of responding, which may differ markedly between animals and conditions. This was an important consideration when making comparisons between saline and METH groups, because METH often caused a large increase in the absolute rates of responding.

Statistical Analysis

The same data analysis plan was followed for both Experiments 1 and 2. The analysis used in Experiment 1 is described here. In order to compare responding during the pre-exposure phase to responding in the test phase, responding during the last two sessions of the pre-exposure phase was compared to responding during the first two sessions of the test phase using three-factor mixed analysis of variances (ANOVA) with drug (saline, METH) and condition (paired, unpaired) as between-subject factors, and time (pre-exposure phase, test phase) as a within-subject factor. If this analysis produced a significant interaction between time X drug, or time X condition, follow up ANOVAs were used to establish the source of these interactions. Separate two-way between subject ANOVAs with drug (saline, METH) and condition (paired, unpaired) as the two factors were performed on the pre-exposure and test phase data.

In order to examine the effects of repeated testing during the 10 session test phase, an analysis was performed to determine the effects of both the drug and pre-exposure condition on active, inactive, and the RFActive during the 10 session test phase. A three-factor mixed ANOVA with drug (saline, METH) and condition (paired, unpaired) as between-subject factors, and time (five two-session blocks) as a within-subject factor was used for this analysis. The source of significant interactions produced by this analysis was followed up with a two-way between-subject ANOVA with drug (saline, METH) and condition (paired, unpaired) as the factors. This follow up analysis was performed on the average of the 10 test sessions. Changes in active responding across blocks were examined by determining if there was a significant difference between the first two-session block of the test phase and the last two-session block of the test phase using within-subject t-tests.

For all statistical tests an alpha criterion of p < 0.05 was used. In cases where t-tests were used to determine the sources of significant interactions, the significance level was adjusted using Bonferroni corrections. The same analysis used in Experiment 1 was performed in Experiment 2, but with NoLt and LT rather than paired and unpaired as the two levels of the condition factor.

Results

Experiment 1: Conditioned Reinforcement

Rats in the paired group demonstrated a significant increase in center responding during the pre-exposure phase when the light was presented. The percent of trials in which the rats poked their snouts into the center hole (water hole) during the 5 s period immediately preceding light onset and during the 5 s light presentation was determined. The same measures, using a comparable time frame, were made for rats in the unpaired condition. Fig. 2 shows that rats in the paired group demonstrated a significant increase in center snout poking (t(26) = 21.709, p < 0.001) when the light was presented. This indicates that rats in this group learned the association between light and water. In contrast, rats in the unpaired group, which also received water in the center snout poke hole, showed a significant decrease in center responding when the light was presented (t(24) = −5.093, p < 0.001). This decrease may reflect a tendency for the rats to localize or orient toward the light, which was located away from the center hole on the ceiling of the test chamber. Rats in the unpaired group were more likely to make a center snout poke response when the light was not present than rats in the paired group. This may have been because there was no signal to inform the rats in the unpaired group about the presence of water in the center hole.

Fig. 2.

The plot shows that rats in the paired group were significantly more likely to snout poke into the center hole when the light was presented than when it was not. In contrast, the unpaired group responded significantly less when the light was presented. Dark gray histogram bars indicate snout poking during the 5 s period prior to light onset. Light gray histogram bars indicate snout poking during the 5 s period of light. Asterisks (*) indicate a significant difference between snout poking during the 5 s period immediately preceding light onset and during the 5 s light presentation. The data are expressed as means and standard error of the means.

Comparison of responding during pre-exposure phase with responding in the test phase

For active responding (Fig. 3, top-left) the three-way ANOVA produced significant interactions between time X drug [F(1,48) =18.73 , p < 0.001] and time X condition [F(1,48) = 40.05 , p < 0.001]. Follow up ANOVAs indicated that there were no significant differences between the four groups during pre-exposure and that there were significant main effects of both drug [F(1,51) = 19.76 , p < 0.001] and condition [F(1,51) = 106.20 , p < 0.001] during test phase. This pattern of results indicates that during the test phase METH increased active responding in both the paired and unpaired groups, and that rats in the paired condition had higher rates of active responding than rats in the unpaired groups.

Fig. 3.

These plots depict active (top), inactive (middle), and the relative frequency of active responding (bottom) for the paired and unpaired groups. The left column depicts the last two-sessions of the pre-exposure phase and responding during the first two-sessions of the test phase, the middle column depicts responding during the testing phase, and the histograms in the right column show the 10-session averages of responding during the testing phase. Each data point represents the average number of responses made per session, averaged over two sessions. Circles indicate responding in the paired group, and squares indicate responding in the unpaired group. Light gray symbols indicate rats treated with saline, and dark gray symbols indicate rats treated with 0.5 mg/kg methamphetamine (METH). Pound (#) signs indicate significant differences between saline and METH treated rats, and ampersats (@) indicates significant differences between the paired and unpaired groups. In the middle column, asterisks (*) indicate significant differences between the first two-session block of testing and the last two-session block of testing. The data are expressed as means and the standard error of the means.

For inactive responding (Fig. 3, middle-left) the three-way ANOVA produced a significant interaction between time X drug [F(1,48) = 45.62 , p < 0.001]. Follow up ANOVAs indicated that there were no significant differences between the four groups during pre-exposure and that there was a significant main effect of drug [F(1,51) = 21.12 , p < 0.001]. This pattern of results indicates that during the test phase METH increased inactive responding in both the paired and unpaired groups and that there was no effect of condition on inactive responding.

For the RFActive (Fig. 3, bottom-left) the three-way ANOVA produced a significant interaction between time X condition [F(1,48) = 62.60 , p < 0.001]. There were no significant effects of drug. Follow up ANOVAs indicated that there were no significant differences between the four groups during pre-exposure and that there was a significant main effect of condition [F(1,51) = 64.12 , p < 0.001] during test. This pattern of results indicates that during the test phase, rats in the paired condition had a higher RFActive than rats in the unpaired condition and that METH did not significantly affect the RFActive.

Responding during the test phase

Table 1 lists the number of response-contingent light presentations earned for active responding during the test phase of Experiment 1. For active responding (Fig. 3, top) the three-way ANOVA produced significant interactions between time X condition [F(4,192) = 7.69 , p < 0.001] and time X drug [F(4,192) = 8.03 , p < 0.001]. Follow up analysis produced significant main effects of drug [F(1,51) = 94.01 , p < 0.001] and condition [F(1,51) = 130.40, p < 0.001] with no significant interaction. This analysis indicates that METH increased active responding in both the paired and unpaired groups, and that rats in the paired condition had higher rates of active responding than rats in the unpaired groups. Within-subject t-tests produced a significant difference between the first and last two-session blocks of the test phase for the paired METH (t(26) = −3.40, p < 0.01) and unpaired METH (t(26) = 3.99, p < 0.01) groups. This pattern of results indicates that the METH-induced increases in active responding became larger with repeated testing.

Table 1.

This table lists the mean (bold) and standard deviation of the mean (italics) of response-contingent light presentations presented according to a variable-interval 2 min schedule during the test phase of Experiment 1. (Each datum represents the average number of light presentations per session, averaged over two sessions.)

Pre-exposure Group		Test Phase Blocks
		1&2	3&4	5&6	7&8	7&10
Saline
Paired		10.11	10.57	9.29	9.54	9.39
		1.64	2.23	1.72	1.74	2.20
Unpaired		5.17	4.63	5.38	5.96	5.38
		2.67	1.57	1.93	1.85	1.98
METH
Paired		11.58	11.69	12.35	12.96	12.85
		1.84	1.88	2.00	2.09	1.65
Unpaired		7.92	9.35	10.58	10.69	11.58
		2.75	2.35	1.75	2.10	1.54

For inactive responding (Fig. 3, middle), the three-way ANOVA produced a significant interaction between time X drug [F(4,192) = 5.15 , p < 0.01]. Follow up analysis produced a significant main effect of drug [F(1,51) = 38.96 , p < 0.001] only. This analysis indicates that METH increased inactive responding in both the paired and unpaired groups and that there was no effect of condition on inactive responding. Within-subject t-tests produced a significant difference for only the paired saline group (t(26) = 3.33, p < 0.01). This significant difference reflects a decrease in responding from the first two-session block of testing to the last two-session block. This pattern of results indicates that repeated testing with METH did not significantly increase inactive responding.

For the RFActive (Fig. 3, bottom) the three-way ANOVA produced significant interactions between time X drug [F(4,192) = 3.13, p < 0.05] and time X condition [F(4,192) = 2.84, p < 0.05]. Follow up analysis produced a significant main effect of condition [F(1,51) = 42.97 , p < 0.001] only. This analysis indicates that rats in the paired condition had a higher RFActive than rats in the unpaired condition and that METH did not significantly affect the RFActive. Within-subject t-tests produced no significant differences between the first and last two-session blocks of testing. This pattern of results indicates that repeated testing with METH did not significantly affect RFActive.

To summarize, response-contingent light was a reinforcer in the paired group. The absolute and relative frequency of active responding was increased by response-contingent light during the test phase in both the paired and unpaired groups, compared to responding during the pre-exposure phase (Fig. 3 left). These increases in responding were maintained throughout the 10 test sessions (Fig. 3 center). Rats in the paired group emitted higher absolute and relative rates of active responding than rats in the unpaired group indicating that light onset was more effective as a reinforcer in the paired group. The rates of inactive responding in the paired and unpaired groups were not significantly different (Fig. 3 middle). METH increased the absolute rate of both active and inactive responding in both the paired and unpaired groups. METH did not significantly increase the RFActive compared to saline in either the paired or unpaired groups (Fig. 3 bottom). With repeated testing the rate of active responding increased across days in the METH treated paired and unpaired groups (Fig. 3).

Experiment 2: Sensory Reinforcement

Comparison of responding during pre-exposure phase with responding in the test phase

For active responding (Fig. 4, top-left) the three-way ANOVA produced significant interactions between time X drug [F(1,46) =33.44 , p < 0.001] and time X condition [F(1,46) = 57.88 , p < 0.001]. Follow up analysis indicated that there were no significant differences between the four groups during pre-exposure and that for the test phase data there were significant main effects of both drug [F(1,49) = 28.46 , p < 0.001] and condition [F(1,49) = 41.64 , p < 0.001]. This pattern of results indicates that during the test phase, METH increased active responding in both the NoLt and LT groups, and that rats in the NoLt condition had higher rates of active responding than rats in the LT group.

Fig. 4.

These plots depict active (top), inactive (middle), and the relative frequency of active responding (bottom) for the NoLt and LT groups. The left column depicts the last two-sessions of the pre-exposure phase and responding during the first two-sessions of the test phase, the middle column depicts responding during the testing phase, and the histograms in the right column show the 10-session averages of responding during the testing phase. Each data point represents the average number of responses made per session, averaged over two sessions. Circles indicate responding in the NoLt group, and squares indicate responding in the LT group. Light gray symbols indicate rats treated with saline, and dark gray symbols indicate rats treated with 0.5 mg/kg methamphetamine (METH). Pound (#) signs indicate significant differences between saline and METH treated rats, and ampersats (@) indicates significant differences between the NoLt and LT groups. In the middle column, asterisks (*) indicate significant differences between the first two-session block of testing and the last two-session block of testing. The data are expressed as means and the standard error of the means.

For inactive responding (Fig. 4, middle-left), the three-way ANOVA produced a significant interaction between time X drug [F(1,46) = 72.72 , p < 0.001]. Follow up analysis indicated that there were no significant differences between the four groups during pre-exposure, and that for the test phase data there was a significant main effect of drug [F(1,49) = 16.28 , p < 0.001] only. This pattern of results indicates that during the test phase, METH increased inactive responding in both the NoLt and LT groups and that there was no effect of condition on inactive responding.

For the RFActive (Fig. 4, bottom-left), the three-way ANOVA produced a significant interaction between time X condition [F(1,46) = 21.71 , p < 0.001]. There were no significant effects of drug. Follow up analysis indicated that there was a significant difference between the saline and METH groups during pre-exposure [F(1,49) = 9.36 , p < 0.01], however for the test phase there was a significant main effect of condition [F(1,49) = 29.32 , p < 0.001] only. A difference between the METH and saline groups during pre-exposure was unexpected since the rats were not treated with drug and most likely reflects a chance difference in baseline performance. However, since this effect was not present during the test phase, it did not impact the results indicating that rats in the NoLt condition had a higher RFActive than rats in the LT condition and that METH did not significantly affect the RFActive.

Responding during the test phase

Table 2 lists the number of response-contingent light presentations earned for active responding during the test phase of Experiment 2. For active responding (Fig. 4, top), the three-way ANOVA produced a significant main effect of drug [F(1,46) = 58.24 , p < 0.001], and an interaction between time X condition [F(4,184) = 3.01 , p < 0.05]. Follow up analysis produced a significant main effect of drug [F(1,49) = 58.24 , p < 0.001] and condition [F(1,49) = 41.81 , p < 0.001] with no significant interaction. This analysis indicates that METH increased active responding in both the NoLt and LT groups, and that rats in the NoLt condition had higher rates of active responding than rats in the LT groups. Within-subject t-tests produced a significant difference between the first and last two-session blocks of the test phase for the LT METH (t(26) = 4.69, p < 0.001) group. This pattern of results indicates that METH-induced increases in active responding in the LT METH group became larger with repeated testing.

Table 2.

This table lists the mean (bold) and standard deviation of the mean (italics) of response-contingent light presentations presented according to a variable-interval 2 min schedule during the test phase of Experiment 2. (Each datum represents the average number of light presentations per session, averaged over two sessions.)

Pre-exposure Group		Test Phase Blocks
		1&2	3&4	5&6	7&8	7&10
Saline
NoLT		7.96	7.96	8.12	8.38	8.88
		2.43	2.98	2.95	2.94	2.49
LT		4.54	4.62	4.92	5.19	5.27
		3.00	2.71	3.01	2.44	2.96
METH
NoLT		11.75	12.46	11.83	12.29	11.75
		2.04	1.54	1.99	1.51	2.97
LT		9.42	10.92	10.58	10.79	11.63
		2.95	1.41	2.56	2.28	1.65

For inactive responding (Fig. 4, middle), the three-way ANOVA produced a significant main effect of drug [F (1,46) = 9.84 , p < 0.01] only. This analysis indicates that METH increased inactive responding in both the NoLt and LT groups and that there was no effect of condition on inactive responding.

For the RFActive (Fig. 4, bottom), the three-way ANOVA produced a significant three-way interaction between time X drug X condition [F(4,188) = 2.55 , p < 0.05]. Follow up analysis produced a significant main effect of condition [F(1,49) = 18.01 , p < 0.001] only. This analysis indicates that rats in the NoLt condition had a higher RFActive than rats in the LT condition and that METH did not significantly affect the RFActive. Within-subject t-tests produced a significant difference between the first and last two-session blocks for the LT METH group (t(26) = 4.22, p < 0.001). This pattern of results indicates that METH-induced increases in the RFActive in the LT METH group became larger with repeated testing.

In summary, response-contingent light was a SR in the NoLt group. Comparisons of responding during pre-exposure to responding during the test phase indicated that the absolute and relative frequency of active responding was increased by response-contingent light in both the NoLt and LT groups (Fig. 4). These increases in responding were maintained throughout the 10 test sessions (Fig. 4). The NoLt group showed the strongest SR effect indicating that pre-exposure to light onset in the LT group decreased the effectiveness of the light as a SR. The NoLt group had higher rates of absolute and RFActive than rats in the LT group. Inactive responding for the NoLt and LT groups was not significantly different (Fig. 4). METH increased the absolute rate of both active and inactive responding in both the NoLt and LT groups. METH did not initially increase the RFActive in either LT or NoLt groups above saline levels (Fig. 4). However, with repeated testing, the absolute rate and the RFActive increased across days in the METH treated LT group (Fig. 4). Similar increases were not observed in the saline treated group or in either the saline or METH treated NoLt groups.

Discussion

Comparison of conditioned and sensory reinforcement

Consistent with previous reports response-contingent light was a reinforcer in the paired group [36–39]. Also consistent with previous reports from our laboratory [48–50], other recent reports [22, 24, 32], and numerous earlier reports (for reviews see [5–8]), response-contingent light was a SR in the NoLt group. In Experiment 1, the importance of pre-exposure to paired light and water for determining the effectiveness of the light as a CR is demonstrated by comparing performance of the paired and unpaired groups. Although rats in both of these groups received the same number of light and water presentations, the outcome was very different. Pavlovian conditioning in the paired group increased the reinforcing effectiveness of light onset compared to the unpaired group which was pre-exposed to unpaired presentations of water and light. In contrast, the absence of pre-exposure to light in the NoLt group increased the effectiveness of light onset as a SR compared to the LT group which was pre-exposed to light onset.

Previous studies have reported that pre-exposure to sensory stimuli decreases the reinforcing effectiveness of these stimuli [6, 52–54]. Kish and Baron [53] proposed that responsiveness to a novel stimulus decreases with frequent experience of the stimulus, and increases with longer intervals between successive exposures. However, the precise mechanism of this decrease in reinforcing effectiveness is unclear. In Experiment 2, pre-exposure may have reduced reinforcing effectiveness for at least two reasons. First, during the pre-exposure phase, repeated occurrences of light onset may have caused the orienting responses elicited by the light to habituate leading the animals to ignore the now familiar light during the test phase. There is evidence that habituation to the sensory properties of primary reinforcers such as food plays an important role in determining reinforcer effectiveness [55, 56]. In support of a habituation interpretation, we have previously reported that there is rapid within-session habituation of the reinforcing effectiveness of light onset [48–50]. More recently, we have shown that the reinforcing effectiveness of light onset is an inverse function of the frequency of light presentations. We found that rats responded at a higher rate for light onset presented according to a responsecontingent VI 6 min schedule than when presented according to response-contingent FR1 or VI 1 min schedules of reinforcement [57].

Alternatively, pre-exposure to response independent presentation of light may have resulted in learned irrelevance. Animals may have learned to ignore the response-contingent light during the test phase because during the pre-exposure phase they learned that snout poking did not affect light onset [47]. Both the learned irrelevance and habituation interpretations describe possible mechanisms that may mediate the effects of decreasing novelty through pre-exposure. Previous theoretical interpretations of the reinforcing effectiveness of SRs [6, 52–54] and more current work from our lab [48–50], support the habituation interpretation. However, in the current experiment a learned irrelevance interpretation cannot be ruled out.

The lowest rates of responding were observed in Experiment 1 for the unpaired group. This group was pre-exposed to random presentations of light and water. Because the light was not paired with water, we did not expect it to be a CR, but the light may still have been a SR. However, the unpaired group in Experiment 1 produced rates of active responding, and RFActive that were much lower than those observed for the NoLt group in Experiment 2. One explanation for the low levels of responding for light onset in the unpaired group compared to responding in the NoLt group is that the light was no longer novel because rats were pre-exposed to the light. These data provide further evidence indicating that novelty mediates the reinforcing effectiveness of light onset.

In summary, the results generally support the interpretation that novel light onset was a more effective reinforcer than familiar light onset. The results of Experiment 1 indicate that the reinforcing effectiveness of the light onset in the paired experiment was increased by prediction of the biologically important water reinforcer while the results of Experiment 2 indicate that the reinforcing effectiveness of the SR (light onset) was increased by novelty.

Effects of METH

METH has similar effects on responding for both CRs in Experiment 1 and SRs in Experiment 2. The effects of METH on responding for SRs and CRs may be best described as amplifying both active and inactive responding. METH caused large absolute increases in the rate of active responding and smaller absolute increases in the rate of inactive responding for both CRs and SRs. However, analysis of the RFActive indicates that the proportion of active responding was not affected by METH. This is despite the fact that visual inspection indicates that METH caused much larger increases in the rate of active than inactive responding. For example, if saline-treated animals emit 100 active and 25 inactive responses and METH-treated animals emit 200 active and 50 inactive responses, we may conclude that there has been a substantial increase in the absolute rate of active responding. However, the RFActive measure of 0.80 would be the same for both saline- and METH-treated groups, indicating that there was no selective effect of the drug on active responding and that the drug amplified both active and inactive responding equally. We conclude that METH did not differentially increase the reinforcing effectiveness of light onset in either the SR or the CR groups.

An explanation of the absence of an effect on the RFActive measure is that responding directed into both the active and inactive holes produced immediate sensory consequences that reinforced these actions. The non-specific effects observed with METH on active and inactive responding are consistent with the interpretation that METH increases the effects of SRs on responding. In contrast, the non-specific effects of METH are not consistent with an interpretation which emphasizes that the effects of METH are mediated by prediction.

Increases in locomotion and other motor movements observed after administration of psychomotor stimulant drugs such as METH may be due to enhanced reinforcing effectiveness of the sensory consequences that they produce. Tapp [8] suggested that the movements of rats represent species typical investigatory behaviors directed at specific aspects of the test chamber, and that the frequency of each behavior is influenced by its sensory consequences. Increases in the frequency of investigatory behaviors would be predicted if METH increases the reinforcing effectiveness of sensory stimuli. For example, in the present experiment, tactile stimulation produced by snout poking may have been reinforcing. The active snout poke hole was more reinforcing because it produced the additional sensory stimulation provided by light onset. If this were the case, then the non-differential effects of METH could be explained by postulating a general METH-induced increase in the reinforcing strength of SRs. Other support for this interpretation is provided by studies indicating that kinesthetic feedback produced by operant responses, such as lever pressing, can be reinforcing. For example, insertion of a lever into a test cage led to lever depressions despite the lack of a programmed consequence [58]. In another study, rats made more contact with a bar that was movable than one that was rigid, which lead to the conclusion that the kinesthetic consequences of bar pressing alone are reinforcing [7]. METH-induced increases in responding may be due to a general amplification of the immediate sensory effects of SRs associated with both the active and inactive alternatives.

A general amplifying effect is also consistent with a matching law interpretation of the data. The matching law states that the relative frequency of each kind of responding matches the relative frequency of its reinforcement [59]. The matching law can be applied to the experimental procedure described in this paper using the following equation:

$$\text{RFActive}={\mathrm{r}}_{\mathrm{A}}/({\mathrm{r}}_{\mathrm{A}}+{\mathrm{r}}_{\mathrm{I}}+{\mathrm{r}}_{\mathrm{O}})={\mathrm{R}}_{\mathrm{A}}/({\mathrm{R}}_{\mathrm{A}}+{\mathrm{R}}_{\mathrm{I}}+{\mathrm{R}}_{\mathrm{O}})$$

In this equation, r indicates responding and R indicates reinforcers. The associated subscripts A, I, and O respectively indicate active, inactive, and other responses/reinforcements available in the test chamber. In the present procedure, active responding (A) produced light onset plus the non-programmed SRs that may result from snout poking; inactive responding (I) produced only non-programmed sensory reinforcing consequences that may have been associated with snout poking; ‘other’ (O) refers to all other possible responses and reinforcing consequences that may occur in the test chamber. Despite the fact that we are unable to measure the non-programmed sensory consequences of active and inactive responding, we can still use the matching law to provide a theoretical interpretation of the effects of METH. If METH-treatment specifically increases the reinforcing effectiveness of the light onset, then there should have been both an increase in the absolute rate of responding and an increase in the RFActive. However, the finding that METH-treatment increased the absolute rate of responding without changing the RFActive indicates that METH increases the reinforcing effectiveness of all available reinforcers equivalently.

The argument made above, recognizes that if light onset is a SR, it is possible that other sensory stimuli in the experimental chamber also act as SRs for the actions that produce them. If METH increases the reinforcing effectiveness of light onset, it is likely that it also increases the reinforcing effectiveness of other SRs that may be present in the experimental chamber. Further, if METH-induced increases in responding depend upon prediction of biologically important reinforcers, then increases in responding to the inactive alternative would not be expected. The current results are consistent with the interpretation that METH effects are mediated by the immediate sensory consequences of both SRs and CRs.

There are a number of studies that have reported the effects of d-amphetamine on responding for CRs [22, 34, 35, 37, 39–41]. In general, these studies report increases in the absolute rate of active responding and smaller, less consistent increases in the absolute rates of inactive responding. Decisions about the specificity of the effects observed have been based on measures of absolute frequency. These studies have not reported the effects of drug on the RFActive, nor were these studies designed to measure the SR effects of the CS or the possibility of SRs interacting with the effects of amphetamine on responding for CRs.

Limitations

There are several limitations to the experiments described in this paper and at least one alternative explanation that should be mentioned. First, only one dose of METH (0.5 mg/kg) was used. A smaller or larger dose of METH may have produced a change in the RFActive for the paired group. We have recently reported the effects of several doses of METH (saline, 0.125, 0.25, 0.5, and 1.0 mg/kg) on responding for light onset as described in the present paper [48]. We found a dose-dependent increase in the absolute rate of active responding and no effect on RFActive supporting the present results. However, these results do not answer the same question about a CR. Another possible limitation of this study is that the RFActive was so high in the paired and NoLt groups (0.75–0.80) that a “ceiling effect” may have prevented observing a METH-induced increase. A third limitation is that the precise mechanism by which pre-exposure to the light decreases reinforcer effectiveness is unclear. Although we have strong evidence from previous studies [48–50, 57] that pre-exposure decreases the reinforcing effectiveness of light onset through habituation, in the current experiment leaned irrelevancy (see discussion above) may also explain the observed decrease in reinforcing effectiveness.

The research described in this paper was motivated by Redgrave’s hypothesis that phasic dopamine signals evoked by novel environmental stimuli serve to reinforce the actions that immediately precede them, where reinforcement is operationally defined as an increase in the probability of occurrence of the action. However, the results can also be interpreted in terms of incentive value by postulating that light onset has incentive properties and that METH increased the incentive properties of the light. Furthermore, it is plausible to suggest that novel stimuli have greater incentive value than familiar stimuli and that the observed increases in responses to the inactive hole can be explained through an enhancement of the incentive properties of the sensory consequences that this behavior produces. Finally, there is evidence indicating that incentive value is dissociable from reward prediction and is dopamine dependent [60].

Implications for cue reactivity and sensation seeking

Novel light onset acts as an unconditioned stimulus (US) for orienting responses that rapidly habituate [61, 62], and has been repeatedly shown to act as a reinforcer for operant responding (see previous discussion). One reason for classifying visual stimuli as “neutral” or “indifferent” is that pairing these stimuli with important biological reinforcers overwhelms the weak intrinsic SR effects of these stimuli. However, it has been demonstrated that the “weak” intrinsic effects of light onset can influence the outcome of conditioning. For example, the final topography of conditioned responses often reflects the form of the orienting response elicited by the CS. Behavior occurring at the beginning of the CS-US interval often reflects the properties of the orienting response to the CS, while the topography of the behavior occurring toward the end of the CS-US interval is more influenced by the US [63–65]. For example, the tendency to approach a CS during Pavlovian conditioning has been labeled sign tracking [66] and the tendency to approach the US during Pavlovian conditioning has been labeled goal tacking [67].

Recently, it has been hypothesized that individual differences in sign tracking relative to goal tracking may be predictive of vulnerability to drug abuse [68] and the impact of aversive cues [69]. Explanations of sign and goal tracking emphasize the prediction by the CS of a salient biologically important event. The data in this paper suggest that pre-existing individual differences in reactivity to the immediate sensory effects of the CS, irrespective of any acquired predictive properties of the CS, may also provide an explanation of individual differences in sign tracking. In support of this suggestion, we have previously reported that responding to produce light onset is predictive of both locomotor activity in a novel environment and METH selfadministration [49, 50]. In summary, individual differences in orienting responses to sensory stimuli that have intrinsic reinforcing properties such as light onset may predict reactivity to cues that are subsequently paired with more important biological reinforcers and may be an indicator of sensation seeking [49].

Light onset as a predictor

Pairing light onset with a biologically important reinforcer (e.g., food) prevents habituation of orienting responses to light onset or reinstates orienting to the light after it has habituated [64]. These effects correspond to the effects of light onset on DA neurons. The presentation of novel light onset evokes phasic firing of DA neurons and this DA cell firing rapidly decreases with repeated presentation of the light [44, 45]. This decrease in phasic DA neuron firing is prevented if light onset predicts a biologically important reinforcer [44]. Thus, both a novel SR, which elicits orienting, and a familiar CR, which elicits conditioned responses are associated with DA neurotransmission that may be affected by psychomotor stimulants such as METH.

There is evidence that METH may prevent normal decreases (habituation) in DA activity evoked by novel sensory stimuli as they become familiar or even reinstate the capacity of familiar sensory stimuli to evoke DA release. According to Redgrave [45], phasic firing of DA neurons in response to novel light onset is mediated by the superior colliculus (SC), which has direct input from retinal ganglion cells. In support of this, Midgley and Tees [70] reported that lesions of the SC impaired orienting to visual stimuli in rats. Furthermore, these authors reported that d-amphetamine decreased habituation to visual stimuli, and reinstated orienting in rats with lesions of the SC. Di Chiara [71] has reported that DA release caused by drug reinforcers such as METH prevents the nucleus accumbens (NAC) shell from habituating to drug associated stimuli. According to these authors, DA release evoked by novel food reinforcers in the NAC shell normally decreases with repeated presentations; however, when a drug reinforcer such as METH is given concurrently with the novel food, DA release in the NAC shell does not habituate. Recently, we have reported that METH-treatment decreased both within- and between-session habituation of light-contingent responding [48]. Together, these results suggest that psychomotor stimulants may disrupt habituation to the usually transient effects of novel sensory stimuli. Although novel light onset is often described as neutral, it has a variety of neural and behavioral effects, which may be amplified and extended by psychomotor stimulants such as METH. These effects do not depend on the ability of the light onset to predict a biologically important reinforcer.

Conclusion

The present results support and extend the results of Winterbauer and Balleine [22] that the effects of psychomotor stimulants such as METH and d-amphetamine on responding for both SRs and CRs can be explained as affecting the immediate sensory consequences of light onset, rather than the prediction of future biologically important consequences. The present results indicate that both SRs and CRs may involve DA mediation. The finding that decreasing novelty of light onset through pre-exposure also decreased its effectiveness as a SR is consistent with Redgrave’s hypothesis that phasic DA release evoked by novel sensory stimuli increases the probability of occurrence of the behavior that produced novel sensory stimulation. Sensory reinforcement may be a DA mediated mechanism through which the organism learns about the sensory consequences of its actions. Understanding the behavioral and neural processes that underlie sensory reinforcement may be important for understanding the effects of psychomotor stimulants on behavior.

Understanding the behavioral and neural processes that underlie sensory reinforcement may be important for understanding the effects of psychomotor stimulants on behavior.

Highlights.

• ▶Novel light onset is a more effective reinforcer than familiar light onset.
• ▶Meth increases responding but not preference for conditioned or sensory reinforcers.
• ▶Meth may enhance immediate sensory rather than predictive properties of stimuli.
• ▶The effects of meth on sensory reinforcement may be mediated by DA.
• ▶Reinforcing effectiveness of abused drugs may be mediated by sensory reinforcers.

Abbreviations

ANOVA

analysis of variance

CS

conditioned stimulus

CR

conditioned reinforcer

DA

dopamine

METH

methamphetamine

NAC

nucleus accumbens

RFActive

relative frequency of active responding

SC

superior colliculus

SR

sensory reinforcer

VI

variable-interval

VT

variable-time

US

unconditioned stimulus