Amount of training and cue-evoked taste-reactivity responding in reinforcer devaluation

Abstract

In two experiments, rats received minimal (16) pairings of one auditory conditioned stimulus (CS) cue with a sucrose reinforcer, and extensive (112) pairings of another auditory CS with that reinforcer. After sucrose was devalued by pairing it with lithium chloride in some rats (Devalue groups) but not others (Maintain groups), taste reactivity (TR) and other responses to unflavored water were assessed in the presence of the auditory CSs alone. The minimally-trained CS controlled substantially more evaluative TR responses than the extensively-trained CS. Those TR responses were hedonic (positive) in the Maintain groups, but aversive (negative) in the Devalue groups. By contrast, food cup entry and other responses thought not to reflect evaluative taste processing were controlled more by the extensively-trained cue. These responses were reduced by sucrose devaluation to a similar extent regardless of the amount of training. The results suggest rapid changes in the content of learning as conditioning proceeds. Early in training, CSs may be capable of activating pre-evaluative processing of an absent food reinforcer that includes information about its palatability, but that capability is lost as training proceeds.

Considerable evidence shows that the contents of associative learning change over the course of extended training. Typically, learned performance is thought to be more flexible and goal-oriented in early stages of training, but becomes increasingly automatic and less governed by its consequences with more extended training (e.g., Allport, 1937; Kimble & Perlmuter, 1970; Tolman, 1948.) Adams and Dickinson (1981) suggested that extended training may be accompanied by a shift from the control of behavior by outcome expectancies (stimulus-reinforcer or response-reinforcer associations) to control by stimulus-response associations. Consistent with this view, several investigators have found that after extended instrumental training, responding is less sensitive to reinforcer devaluation (e.g. Adams, 1982; Bussey et al., 1996; but see Colwill & Rescorla, 1985; Holland, 2004), a primary indicant of the mediation of performance by outcome expectancies (Pickens & Holland, 2004). For example, Adams (1982) found that after small amounts of food-rewarded lever press training, devaluation of the food by pairing it with a toxin resulted in spontaneous reductions in lever pressing in the absence of food, whereas comparable food devaluation after extended lever press training left responding unaffected.

Other evidence suggests that the content of learning may change more subtly within the range of training parameters that produce outcome-mediated learning. Holland (1990, 1998) found that although the performance of Pavlovian conditioned responses (CRs) remained sensitive to outcome devaluation even after relatively extended training, a conditioned stimulus (CS) could substitute for its referent unconditioned stimulus (US) in representation-mediated flavor aversion learning only after minimal amounts of training. Holland (1981, 1990) found that food aversions may be established when an associatively-activated representation of food is paired with illness. For example, in one experiment (Holland, 1981), rats first received pairings of two auditory CSs with two food reinforcers that differed only in flavor (e.g., tone → wintergreen-flavored sucrose and noise → peppermint-flavored sucrose), to give those CSs the ability to selectively activate representations of one or the other of those reinforcers. Next, presentations of one of the auditory CSs alone were paired with the injection of the toxin lithium chloride (LiCl), in the absence of any flavors. Subsequent consumption tests showed the establishment of a mild aversion to the food whose CS partner had been paired with illness, as if the CS’s activation of a representation of food just prior to the induction of illness permitted the formation of food-illness associations. Later, Holland (1990, 1998, 2005) found that the establishment of mediated aversions depended on minimal CS-US training. Although CS-LiCl pairings established significant mediated food aversions after 16–24 CS-food pairings, rats trained with 40 or more CS-food pairings showed no evidence for mediated food aversions. At the same time, other rats trained with as many as 160 CS-US pairings continued to display evidence for outcome encoding, as measured by reinforcer devaluation procedures; that is, pairings of food itself with illness reduced subsequent responding to the associated CS. These results led Holland (1990, 1998, 2005) to suggest that early in training, CSs activate a wide range of processing usually activated by the US itself, including early-stage perceptual processing, but as training continues, that access narrows to more limited sets of processing systems. Thus, within this view, mediated flavor aversion learning was not observed later in training because the CS no longer activated perceptual processing of the absent flavor, which could enter into associations with illness.

The experiments reported here describe a change in another aspect of learning that occurs early in training, CSs’ access to US processing that supports the display of evaluative “taste reactivity” (TR) responses. While consuming fluids, rats often display distinctive TR responses, which reflect motivational or evaluative attributes of those fluids (e.g., Grill & Norgren, 1978). Preferred substances such as sucrose normally elicit positive TRs (e.g., tongue protrusion), whereas substances such as quinine and concentrated salts normally elicit negative TRs (e.g., gapes). Furthermore, these responses seem to be tied to the evaluation of those substances rather than their sensory properties. For example, if an aversion is established to sucrose by pairing it with LiCl, then the positive TR response to sucrose is replaced with a negative one (Berridge et al., 1981; Pelchat et al., 1983). Likewise, the normally negative TR response to concentrated NaCl is replaced by a positive TR if a rat is made sodium deficient (Berridge & Schulkin, 1989). The logic behind the present experiments is that if a tone paired with sucrose activates perceptual or evaluative processing of the absent sucrose, then presentation of the tone in the presence of unflavored water might also elicit positive TR responses. If this associative activation of hedonic processing occurs only early in acquisition, then only a minimally-trained CS would control these positive evaluative responses, and a more extensively-trained CS would not.

The results of a number of early studies suggested that the evaluative, motivational properties of consumed substances could be transferred to CSs by associative learning. For example, Delamater, Berridge, and LoLordo (1986) found that auditory cues paired with sucrose or quinine later could provoke positive or negative TR responses, respectively, when those CSs were presented while unflavored water was infused into the rats’ oral cavities. Delamater et al. (1986) interpreted these data as indicating that the tones had acquired the capacity to alter the palatability of water by evoking detailed representations of the sensory and motivational properties of sucrose and quinine. However, as Berridge and Schulkin (1989) noted, these data might instead reflect simple S-R learning, by which the behavior patterns initially controlled by the USs were transferred to the control of the CSs.

Berridge and Schulkin (1989) used a reinforcer revaluation procedure to demonstrate the acquisition of control of TR responses by CSs under circumstances that made an S-R account unlikely. Rats received pairings (in simultaneous compounds) of one neutral CS flavor (weak acetic acid or quinine HCl) with a strong (0.39M) NaCl flavor, which elicited negative TRs, and the other neutral CS with a sucrose solution. Responding to the CS flavors alone was assessed after rats were made sodium deficient by injection of furosemide. In test, the rats displayed positive TR responses to the CS that had previously been paired with NaCl, consistent with the change in rats’ TR responses to NaCl itself (described earlier). Recall that at the time of the original CS-NaCl pairings, only negative TRs occurred. Thus, only negative TR responses could have been acquired by S-R learning. Instead, the CS accessed the current motivational significance of the reinforcer with which it had been paired.

Later, Holland (1990, exp. 1) and Kerfoot et al. (2007) used a similar rationale to examine the evaluative content of learning when a tone was paired with sucrose. In Kerfoot et al.’s (2007) study, rats in two experimental groups first received, in a single session, 16 pairings of a tone with intra-oral delivery of sucrose, while control rats received those events unpaired. The next day, the rats received a session with sucrose presentations only. A single LiCl injection was administered either immediately after the session (Groups Devalue and Devalue-control) or 8 hours later (Groups Maintain and Maintain-control). Then, all rats received a test that included pairings of the tone with unflavored water. The rats in Group Maintain displayed substantial positive TRs to the tone, especially when unflavored water was infused concurrently, to provide a substrate for TR responses. By contrast, the rats in Group Devalue displayed low levels of positive TRs but substantial negative TRs. Notably, these rats had not previously displayed negative TRs at any time in the experiment. During both the tone-sucrose pairing session and the sucrose-alone devaluation session, only positive TRs were observed (the rats were not made ill until after the devaluation session). Thus, there was no basis for S-R learning of negative TRs in Group Devalue. Finally, the control rats showed little evidence for TR responding.

In the present study, we combined the methods of Holland (1998, 2005) and Kerfoot et al. (2007) to examine changes in the ability of auditory CSs paired with sucrose to activate evaluative processing of the sucrose. As in Holland’s (2005) mediated taste aversion and devaluation studies, in each experiment, the rats first received 16 pairings of one auditory CS with sucrose (“minimal training”) and 112 pairings of another CS with sucrose (“extensive training”), randomly intermixed within 8 sessions. We predicted that although the extensively-trained CS would control more hedonically neutral CRs such as food cup approach and anticipatory licking responses than the minimally-trained CS, the minimally-trained CS would control more hedonically positive responses. Next, in half the rats in each experiment, the sucrose was devalued by pairing it with LiCl, while sucrose’s positive value was maintained in the remaining rats, which received unpaired presentations of sucrose and LiCl. We predicted that sucrose devaluation would be reflected in the display of negative TR responses only to the minimally-trained CS. At the same time, we expected sucrose devaluation would decrease the frequency of hedonically-neutral appetitive and consummatory CRs, such as food cup approach and simple licking, to both CSs, as well as eliminate positive TR responses to the minimally-trained CS. In Experiment 1, sucrose was delivered to a cup in standard conditioning chambers, whereas in Experiment 2, sucrose was delivered through a chronic oral cannula, as in Kerfoot et al.’s (2007) study.

Experiment 1

Methods

Subjects

The subjects were 38 male naive Long-Evans strain rats (Charles River Laboratories, Raleigh, NC, USA) which weighed between 300–350 g when they arrived in the laboratory vivarium. They had free access to lab chow (2018 Rodent Diet, Harlan Teklad Laboratory, Madison, WI, USA) for a week before their food was restricted to maintain them at 85% of their ad-libitum weights. The rats were caged individually in a colony room illuminated from 6:00 am to 6:00 pm. One rat became ill prior to the test phase; its data from all phases were discarded. The research protocols were approved by the Johns Hopkins University Animal Care and Use committee.

Apparatus

There were 8 training chambers (22.9 × 20.3 × 20.3 cm) with aluminum front and back walls and clear acrylic side walls and top. An infrared activity monitor (Coulbourn Instruments, Allentown, PA, USA) and a panel of infrared lights used to illuminate the chamber for video recording were placed on the top of each chamber. An illuminated clear acrylic food cup, with a capacity of about 1.7 ml, was placed behind a square hole in the center of the front wall. A photocell beam in the food cup was used to detect head entries and time spent in the cup. Aluminum boxes (3.0 × 2.0 × 3.0 cm), covering levers, were mounted on each side of the food cup, centered between the cup and the side walls. A speaker which was used to present auditory cues was placed on the back wall of a double-walled sound-resistant shell, which enclosed each experimental chamber. A television camera to record the rat’s overall behavior was placed 18 cm above the speaker, and a second camera to record consummatory responses was placed under the transparent food cup. Images from each camera were digitized, and were recorded and displayed in real time on video monitors, each of which showed 4 chambers or food cups.

Procedures

Table 1 provides an outline of the procedures of Experiment 1. All rats first received two 64-minute sessions designed to train them to approach the food cup and consume the sucrose unconditioned stimulus (US). Each of these sessions included 16 deliveries of the reinforcer used throughout the study, 0.1 ml of 0.2M sucrose solution, administered over a 1-s interval by infusion pumps located outside the double-walled sound-attenuating shells. Next, rats in the Maintain and Devalue groups were given 8 32-min training sessions intended to extensively train one auditory CS and minimally train another auditory CS; rats in the Unpaired control group received the same number of presentations of each CS, but explicitly unpaired with sucrose deliveries. In each session there were 14 presentations of one CS and 2 presentations of the other CS, randomly intermixed, with variable intertrial intervals (range 1–3 min). The two CSs were a 78-dB, 1500-hz tone and an 80-dB white noise, each 11s in duration, with the last 1 s overlapping sucrose delivery in the Maintain and Devalue groups. The identities of the two CSs were counterbalanced.

Table 1.

Outline of Procedures

Group	n	Training	Devaluation	Test
Devalue	8	(16) toneM → sucrose	sucrose → LiCl	toneM → water
Minimal		(112) toneE → sucrose
Devalue	8	(16) toneM → sucrose	sucrose → LiCl	toneE → water
Extensive		(112) toneE → sucrose
Maintain	7	(16) toneM → sucrose	sucrose/LiCl	toneM → water
Minimal		(112) toneE → sucrose
Maintain	6	(16) toneM → sucrose	sucrose/LiCl	toneE → water
Extensive		(112) toneE → sucrose
Unpaired	4	(16) toneM/sucrose	sucrose → LiCl	toneM → water
Minimal		(112) toneE/sucrose	sucrose/LiCl
Unpaired	4	(16) toneM/sucrose	sucrose → LiCl	toneE → water
Extensive		(112) toneE/sucrose	sucrose/LiCl

Notes. → signifies paired presentations and/signifies unpaired presentations. The subscripts “M” and “E” signify minimal and extended training; the numbers in parentheses indicate the total number of trials of each type received over the 8 training sessions. On the devaluation days, the rats in the Devalue groups received lithium chloride (LiCl) injections immediately after the sucrose session and the rats in the Maintain groups received LiCl injections 6–8 hr after the sucrose session. In the Devaluation phase, half of the rats in each of the Unpaired subgroups received Devalue training and half received Maintain training. The designs of Experiments 1 and 2 were similar except that in Experiment 2 (1) all fluids were delivered via oral cannulae (2) there was no unpaired training group and (3) each rat was tested with both tones, in counterbalanced order (thus there were only two groups, Devalue and Maintain).

The rats then received sucrose devaluation training over the next 6 sessions. In sessions 1, 3, and 5, the rats received 16 unsignaled sucrose presentations over the course of each 32-min session in the experimental chambers. Each of the rats in Group Devalue and half of the rats in the Unpaired group were injected with 5 mg/kg 0.3-M LiCl as it was removed from the experimental chamber, whereas the rats in Group Maintain and the other half of the Unpaired group received a similar injection 6–8 hr later. In sessions 2, 4, and 6, the rats were placed in the chambers for 32 min but only dummy, empty trials were presented. The rats did not receive injections after these sessions.

Finally, all rats received a 16-min test session that included 8 presentations of one of the CSs (completely counterbalanced for Group, amount of training, and CS identity). The final 1-s interval of the 11-s CS was accompanied by the delivery of deionized water (rather than sucrose) to the food cups. Half of the rats also received a second test session, in which responding to the other CS was assessed. (The other half of the rats were sacrificed for evaluation of immediate early gene activity, but because of a freezer failure the tissue was lost prior to analysis). However, responding in this test was substantially lower than in the first test, making within-subject comparisons difficult. Thus, we do not comment further on this test.

Behavioral analysis

The time spent in the food cup and the number of food cup entries were recorded automatically by photocell circuits. We reported these measures in the last 5-s auditory CS period prior to sucrose delivery on each trial in training sessions, during the 5-s intervals initiated by sucrose deliveries in the LiCl-paired devaluation sessions, and during the corresponding dummy trial periods in the context-only sessions in the devaluation phase. These automated measures were also reported for the 10-s baseline interval prior to each trial in each phase of the experiment.

TR responding in the food cups was recorded and tapes were scored for positive and negative responses. We report responding during the devaluation and test phases only. Behavioral scoring in the devaluation sessions was divided into pre-trial (10 seconds before trial start) and post-sucrose (the 5-s interval initiated by sucrose presentation) periods. In the test sessions, the observation periods included 10-s pre-trial, 10-s CS-alone (before water delivery), and 5-s post-fluid (initiated by water delivery) intervals. Tapes were viewed at 1/10 normal speed.

The only positive behavior comparable to those described by Berridge (2000, in rats with intraoral cannulas), that we observed within the recessed food cups with appreciable frequency, was tongue protrusion. For this behavior, the tongue was widely spread along the floor of the food cup while licking, maintaining contact with the cup floor for most of the lick duration, and completely concealing the rats’ teeth. We counted the number of these licks (“long licks”) as well as the number of licks without tongue protrusions (“short licks”). Short licks included contact with the cup floor but the tongue was not spread. These two categories of licking were readily distinguishable in slow motion; agreement between two observers was nearly 100%. Although Holland (1990) reported the display of a range of aversive behaviors in a somewhat similar apparatus in which rats drank fluids freely, we failed to see significant numbers of these behaviors in this experiment, except for an altered lick topography. Thus, we report only “aversive licks”, in which the rat barely made contact with the solution surface and the tongue maintained a very narrow profile. Bout lengths for this behavior were typically quite short, consistent with the research of Davis (e.g., Davis, 1988). Although these aversive licks were readily distinguishable from positive long licks (100% observer agreement), there was some overlap in judgments of aversive and short licks (80–85% agreement between two observers).

Results

Training

Figure 1 shows food cup responding during the two CSs and during pre-CS periods in the training phase. All rats in Groups Devalue and Maintain quickly learned to approach the food cup during presentation of the auditory CSs. Responding was consistently higher during the extensively-trained cue than during the minimally-trained cue in those groups, but not in the Unpaired control group.

Figure 1.

Acquisition of food cup conditioned responses in the training phase of Experiment 1. Ext refers to responding during the extensively-trained cue, which was presented 14 times in each session, Min signifies responding during the minimally-trained cue, which was presented twice in each session, and pre indicates responding during the baseline intervals prior to each cue presentation.

A group (Devalue, Maintain, or Unpaired) X cue counterbalancing X amount of training (2 or 14 trials/session) X sessions ANOVA of responding during the CSs showed significant effects of group, F(2, 31) = 36.89, p < 0.001, amount of training, F(1, 31) = 18.95, p < 0.001, and sessions, F(7, 217) =31.28, p < 0.001. In addition, the group X amount of training, F(2, 31) = 4.11, p = 0.026, and group X sessions, F(14, 217) = 9.17, p < 0.001, interactions were significant. Those interactions were nearly entirely due to differences between the unpaired group and the other two groups; neither interaction was significant when the unpaired group was excluded from the analysis, Fs < 1.08. Pre-CS responding did not differ among the groups; a group X sessions ANOVA showed no effect of group or group X sessions interaction, Fs < 1.

Devaluation

Figure 2 shows food cup responding during the devaluation sessions. The rats that received injections of LiCl immediately after sessions that included unsignaled sucrose presentations (Group Devalue and the devalued rats in Group Unpaired), gradually reduced food cup entries during those sessions, whereas the rats that received LiCl injections delayed 6–8 hr (Group Maintain and the devalued rats in Group Unpaired), did not. Food cup entries during empty sessions did not differ between the groups. A group X session ANOVA for sucrose sessions showed significant effects of devaluation treatment, F(1, 35) = 4.68, p = 0.037, and session, F(2, 70) = 9.77, p < 0.001, and a significant devaluation X session interaction, F(2, 70) = 10.69, p < 0.001.

Figure 2.

Food cup responding during the 5-s intervals after unsignaled sucrose deliveries and corresponding empty periods in the devaluation phase of Experiment 1. Rats in the Devalue condition received injections of lithium chloride (LiCl) immediately after each of three sucrose devaluation sessions, and rats in the Maintain condition received those injections 6–8 hr later (circles). In each of three empty sessions (inverted triangles), all rats were placed in the chambers and neither sucrose nor LiCl administered.

Figure 3 shows the number of licks observed in the 5-s periods after sucrose deliveries in the last half of the final devaluation session, as scored from the video tapes. The rats in the Devalue condition showed more aversive licks, F(1, 35) = 93.15, p < 0.001, but fewer positive long licks, F(1, 35) = 74.89, p < 0.001, than the rats in the Maintain condition. Rats in the two conditions did not differ in their display of short licks, F < 1. No other aversive TR responses were evident. However, the rats did not spend the entire post-sucrose periods in the food cup (see Figure 2), and so it is possible that gapes, head-shaking and other responses occurred when the rats’ mouths were not in view of the cup cameras.

Figure 3.

Taste reactivity responses to sucrose delivery in the final devaluation session of Experiment 1. Rats in the Devalue condition received lithium chloride injections immediately after each session whereas rats in the Maintain condition received those injections 6–8 hr later. Aversive licks involved a narrowed tongue, barely making contact with the liquid; teeth were always visible. Long (positive) licks involved a broadened and protruding tongue, obscuring the teeth, and long-duration contact with the substrate. Short licks involved an intermediate (normal) tongue breadth, extension, and contact with the substrate. See Behavioral analysis for details.

Test

Figure 4 shows food cup responding of the rats in Groups Maintain and Devalue during the 5-s pre-CS periods, the final 5-s CS periods, and the 5-s periods initiated by water delivery in the test session. We portray the results of the first (top panel) and second (bottom panel) halves of the test session separately, to facilitate comparisons with TR responses, which extinguished much more rapidly (see below). Food cup responding during both CS-alone and CS+water periods was significantly greater in Group Maintain than in Group Devalue, but was unaffected by the amount of training. Food cup responding occurred at very low levels in Group Unpaired (< 10% in each condition, overall mean ± sem = 4.5 ± 2.5%), and was unaffected by either the amount of training or whether sucrose had been paired or unpaired with illness. Pre-CS responding did not differ across groups or trial types.

Figure 4.

Food cup responding on trials with the minimally- and extensively-trained conditioned stimuli (CSs) in in the test session of Experiment 1. The original sucrose reinforcer had been devalued by immediate pairings with lithium chloride injections in the Devalue (Dev) groups, but its value was maintained in the Maintain (Main) groups. Responding is shown separately for the successive periods prior to the CS (pre-CS), during the CS alone, and during the CS while water was delivered to the food cup. The top panel shows responding in the first half of the test session and the bottom panel shows responding in the second half of the session.

Because of the much lower response levels in Group Unpaired, the presence of only 2 rats per test stimulus and devaluation condition in that group, and the lack of systematic differences in responding among these control test conditions, responding of that group was excluded from analysis of the test data. For the remaining rats, a devaluation group (maintain or devalue) X amount of training X CS period (CS alone or CS+water) X test block (first vs second) ANOVA of food cup responding showed significant effects of devaluation, F(1, 25) = 13.82, p = 0.001, CS period, F(1, 25) = 7.91, p = 0.009, and block, F(1, 25) = 17.02, p < 0.001, and an effect of amount of training that bordered significance, F(1, 25) = 3.63, p = 0.068. None of the interactions was significant, Fs < 1. Separate devaluation group X amount of training ANOVAs of food cup responding in groups Maintain and Devalue for each sampling period and each block of testing showed significant effects of devaluation for both CS-alone, Fs(1, 25) > 8.72, ps < 0.007, and CS + water periods, Fs(1, 25) > 5.13, ps < 0.033, but no significant interactions of devaluation group with amount of training, Fs < 1. The effects of amount of training were significant only in the second block of testing, Fs(1, 25) > 4.26, ps < 0.048. ANOVAs of pre-CS responding showed no effects or interactions, Fs < 1.

Figure 5 shows lick topography during the first half of the test session. TR responses extinguished rapidly, as they have been observed to do in the case of taste aversions themselves (Cantora et al., 2006), such that there were essentially no TR responses observed in the second half of the test session. Unlike in the devaluation session, which included sucrose itself, only 3 rats (all in Group Devalue) displayed aversive licks or any other aversive response (Berridge, 2000). Furthermore, few (< 5) licks of any kind were observed during the CS alone, prior to water delivery, in Groups Devalue or Maintain, or during any observation interval in Group Unpaired. Thus, Figure 5 shows only positive (long) and short licks during the CS+water test periods, in Groups Maintain and Devalue. As predicted, the rats in group Maintain tested with the minimally-trained CS showed more positive licks than the rats in that group that were tested with the extensively-trained CS. By contrast, the extensively-trained CS controlled more short licks than the minimally-trained CS in that group. As noted with food cup entry, sucrose devaluation had comparable effects on short licks for both the minimally and extensively-trained CS. However, devaluation reduced the frequency of positive licks to the minimally-trained CS more than it reduced the frequency of those licks to the extensively-trained CS.

Figure 5.

Taste reactivity responding to the minimally- and extensively-trained conditioned stimuli in the test session of Experiment 1. Long (positive) licks involved a broadened and protruding tongue, obscuring the teeth, and long-duration contact with the substrate. Short licks involved a normal tongue breadth, extension, and contact with the substrate. The responding displayed is that from the observation period in which the auditory stimuli were accompanied by water. See Behavioral analysis (Experiment 1) for or details. The original sucrose reinforcer had been devalued by immediate pairings with lithium chloride injections in the Devalue (Dev) groups, but its value was maintained in the Maintain (Main) groups.

A group (Maintain or Devalue) X amount of training ANOVA for positive licks showed a significant effect of group, F(1, 25) = 23.30, p < 0.001, and a significant group X amount of training interaction, F(1, 25) = 7.07, p = 0.014. Planned comparisons showed that the minimally-trained CS in group Maintain controlled significantly more positive licks than the extensively-trained CS in that group, p = 0.017, and more than either of the CSs in Group Devalue, ps < 0.001. The extensively-trained CS also evoked more positive licks in Group Maintain than in Group Devalue, p = 0.026. A comparable ANOVA for short licks showed an effect of both group, F(1, 25) = 10.37, p = 0.004, and amount of training, F(1, 25) = 11.23, p = 0.003, but no interaction, F< 1. Individual comparisons showed that the extensively-trained CS controlled more short-licking than the minimally-trained CS in both groups Maintain and Devalue, ps < 0.02, and that the minimally- and extensively-trained CSs each controlled more short licks in Group Maintain than in Group Devalue, ps < 0.038.

Few long or short licks were observed during the CS + water test intervals in Group Unpaired. However, in that group both CSs served as signals for the absence of sucrose during training, and so these rats were unlikely to approach the food cup during the CSs, and liquid delivery itself was soundless. Thus, it is difficult to interpret the absence of TR responses to water in the 5-s CS + water intervals in this group. To explore this question further, we also scanned the video tapes between trials to observe the rats’ responses when they did encounter water in the food cup. Most contacted the water on the initial test trials, but subsequent slow-motion analysis revealed no long licks and only a few (0–19) short licks. Indeed, most of the delivered liquid was still in the food cups when these rats were removed from the chambers. Thus, somewhat surprisingly, the food cup or liquid delivery itself did not seem to serve as a substitute for the taste of sucrose in the same way as the auditory CSs did in the other groups.

Discussion

Consistent with the results of previous studies (e.g., Delamater et al., 1986; Kerfoot et al., 2007), we found that an auditory CS paired with sucrose acquired control over positive TR responses. However, these hedonic responses were considerably more frequent in response to a CS that had been given minimal amounts of training (16 CS-sucrose pairings) than to a CS that received extended training (112 pairings). This outcome is consistent with the view that early in training, CSs gain access to the evaluative properties of the US, but lose that access with extended training. Because each rat received both minimally-trained and extensively trained CSs in each training session, this difference is not attributable simply to the amount of exposure to the US (e.g., Adams & Dickinson, 1981) or to changes in the form of TR responding to the US over the course of training. Furthermore, the performance of control rats that received the two CSs unpaired with the US indicated that the shift in response form we observed with extended training in the other groups did not reflect habituation of unconditioned positive responses to the auditory CSs or some other nonassociative factor.

By contrast, appetitive food cup entry responses increased in frequency with extended training (as assessed in the training phase), as did consummatory short lick responses, which may resemble the passive intraoral consumption described by Berridge (2000) as nonhedonic. Thus, only positive TR responses were inversely sensitive to the amount to training. Nevertheless, food cup and short-lick responses were each sensitive to sucrose devaluation, regardless of the amount of training received. Thus, some sensory and motivational aspects of the reinforcer remained accessible to the CS even after extended training. These results are consistent with those of Holland (1998, 2005) who found that although only minimally-trained CSs could substitute for their US partners in mediated taste aversion learning, extensively-trained CSs that could not participate in mediated learning nevertheless were at least as sensitive to US devaluation as minimally-trained CSs. The implications of rapid changes in the coding of TR responses and in the ability of CSs to participate in mediated flavor aversion learning, concurrent with little or no change in those CSs’ susceptibility to US devaluation procedures, will be considered in the General Discussion.

The results of Experiment 1 differ in some ways from those of prior studies. Delamater et al. (1986) observed positive TR responses to water in the presence of CSs that had been paired with sucrose either 20 (Exp. 2) or 84 (Exp. 1) times. Although the procedures of their two experiments were so different as to preclude legitimate comparisons of the magnitude of hedonic TR responses between them, the observation of any such responses with more extended training in their Experiment 1 differs from our results. However, that experiment differed from ours in several ways. First, it involved longer CS durations and a discriminative conditioning procedure, which may have slowed the rate of learning. Second, although Delamater et al. (1986) trained their rats with sucrose delivered to a food cup, as we did in Experiment 1, they assessed TR responding to water infused intraorally. Thus, their assessment of TR responses might have been more sensitive than ours. Third, as Berridge and Schulkin (1989) noted, in the absence of devaluation or other such procedures, an S-R basis for the TR responses Delamater et al. (1986) observed with more extended training can not be ruled out.

Our results also differed somewhat from those of Kerfoot et al. (2007). First, here we found no evidence for the auditory CS’s control over TR responses until after the unflavored water substrate was provided (CS + water or post-CS periods), whereas Kerfoot et al. (2007) observed low but significant levels of such control during CS presentations even before presentation of the water substrate. Second, we found no evidence of spontaneous transfer of the aversive TR response established to sucrose itself during devaluation training in Group Devalue, to either auditory CS. By contrast, Kerfoot et al. (2007) found substantial display of aversive TRs to a CS after devaluation of its sucrose reinforcer.

Notably, in Kerfoot et al.’s (2007) study, the sucrose US was presented intraorally throughout the experiment, whereas in Experiment 1, sucrose was delivered to a food cup, which required appetitive approach behaviors and generated somewhat different consummatory responses than are observed with intraoral fluid administration. Previous findings from other laboratories suggest that this procedural difference may have contributed to the differences in responding to the minimally-trained CS observed in Experiment 1 compared to Kerfoot et al.’s (2007) study. Delamater et al. (1986), who, as we did, trained their rats with sucrose delivered to a food cup (but tested with intraoral presentation), also saw little evidence for TR responses to their CSs alone, prior to delivery of the water substrate. Furthermore, Colwill and Rescorla (1990) reported that the effects of reinforcer devaluation on operant responding were considerably greater when the reinforcer was presented intraorally throughout training and testing than when it was delivered to a cup. Accordingly, Experiment 2 used training procedures similar to those of Experiment 1, except that the sucrose US was delivered intraorally throughout the experiment.

Experiment 2

Methods

Subjects

The subjects were 8 male Long-Evans rats, obtained and maintained as described in Experiment 1.

Surgical procedures

Aseptic precautions were taken to prevent bacterial infection. Rats were first anesthetized with isoflurane gas (Abbot Laboratories, North Chicago, IL), and an intraoral cannula was implanted just anterolateral to the second maxillary molar on each side, using surgical procedures similar to those described by Grill and Norgren (1978). Each intraoral cannula consisted of PE-100 polyethylene tubing (Fisher Scientific, Pittsburgh, PA) flared at one end and fitted with a washer to hold it in place in the mouth. Each cannula was threaded under the skin to a ½-in segment of 19-gauge hypodermic tubing which served as a connector for fluid delivery. After the rat was placed in a stereotaxic frame (Kopf Instruments, Tujunga, CA) and fitted with an anesthesia cone, this connector was cemented in place on the rat’s skull with dental acrylic, secured with 4 skull screws.

Postoperative care

Immediately after the surgery, rats received an injection of buprenorphin (0.03 mg/kg) to ameliorate pain. Rats were given 10 days to recover after surgery, with daily oral administration of 0.5ml of cephalexin (250mg/5ml, Teva Pharmaceuticals, Sellersville, PA) to prevent infection. After 10 days, cephalexin was discontinued and the rats were placed on their food-restriction diet. The cannulas were flushed daily with deionized water throughout the recovery and experimental periods.

Apparatus

The behavioral training apparatus consisted of two clear PVC (polyvinyl chloride) cylinders (36.8 cm tall and 28 cm in diameter) mounted on top of a clear floor. The apparatus was elevated on a stand measuring 36.8 cm in height. A color CCD camera was placed underneath each clear floor so that the full range of the cylinder was in view and the rats could be seen from below. A speaker (Moisture Resistant Speaker System MX1, Radio Shack, Fort Worth, TX), which delivered 1000 and 4000 Hz tones, was placed in the front left corner of the floor just outside each cylinder. A syringe pump (Razel Scientific Instruments, Inc., Stamford, CT) for delivery of liquids, was positioned at the bottom of each stand.

Procedures

The rats received training, devaluation, and test phases similar to those of Experiment 1 (Table 1). At the beginning of each session, a length of PE-100 tubing extending from a 1-ml syringe in the pump was attached to the intraoral cannula connector on the rat’s head. Separate syringes and lines were used for sucrose and water. In the first session, the rats were adapted to the cylinder and oral infusion procedure by placing them in the cylinder for 16 min with no events and then, over the final 16 min, delivering 8 of the reinforcers to be used in training, 5-s 0.05-ml infusions of 8% sucrose. In each of the 8 subsequent 32-min training sessions, all rats received 14 15-s reinforced presentations of one tone CS and 2 reinforced presentations of the other tone CS. The identities of the two tones (1000 and 4000 hz) were counterbalanced. The last 5 s of each tone was accompanied by sucrose infusion. Although the CS-US interval was 10 s in both Experiments 1 and 2, the CS and US duration was longer in Experiment 2 to accommodate the longer times needed to infuse sucrose directly into the mouth without its being aversive.

The rats then received devaluation training over the course of 4 sessions. In sessions 1 and 3, each rat received 16 5-s sucrose infusions over 16 min, in the absence of tones. Each rat (n=4) in Group Devalue received a 5 ml/kg injection of 0.3M LiCl as it was removed from the cylinder, whereas each rat in Group Maintain (n=4) received that injection in its home cage 6–8 hr later. In sessions 2 and 4, the rats were placed in the cylinders and connected to the infusion lines, but no tones, fluids, or injections were given. Finally, in two separate 8-min test sessions, we administered 4 15-s presentations of each of the two tones, with the last 5 s of each tone accompanied by the infusion of deionized water. Test order was completely counterbalanced across group, amount of training and CS identity. We used a small number of test trials to minimize the problems produced by rapid extinction, observed in Experiment 1, and to permit the use of within-subject comparisons of responding to the minimally- and extensively-trained CSs.

Behavioral analysis

Behavior in each phase was recorded and tapes for the first and last sessions of training, the final devaluation session, and the two test sessions were scored for positive and negative TR responses. All test trials were scored. Responses in the training sessions were scored for the two minimally-trained CSs and for two extensively-trained CSs, one that occurred on the trial before a minimally-trained CS and one that occurred on the trial after a minimally-trained CS (to equate position of the minimally- and extensively-trained CSs within the session). Behavioral scoring for these sessions was divided into pre-trial (10 seconds before trial start), tone-alone (10 s during tone before liquid presentation), tone + liquid (last 5 s of tone presentation, while sucrose [training] or water [testing], was infused), and post-trial (10 seconds after the end of the tone) periods. Scoring for the devaluation session included responding during the 10 s before, 5 s during, and 10 s after the last 8 sucrose infusions. Because the camera view was fixed, on some trials these behaviors could not be observed as a result of the rat’s posture. We did not compensate for missing data-scoring opportunities. Tapes were viewed at 1/10 normal speed. Licks with tongue protrusions and short licks and swallows without tongue protrusions were counted individually, as in Experiment 1. These behaviors were not identical in form to those observed in Experiment 1, in which rats licked liquids from a surface, but were similar to those described by Grill and Norgren (1978) and Berridge (2000) as lateral tongue protrusions and “intraoral intake” responses, respectively. Those authors characterized intraoral intake responses as neutral, neither hedonic nor aversive, similar to our characterization of short licks without tongue protrusions. The only other hedonic response observed with appreciable frequency was paw licking. Because of the non-discrete nature and relatively long bout lengths of that behavior, it was scored using a time-sampling procedure in which a judgment of whether paw licking was in progress was made at each observation. Observations were made each 1.25 s of slow motion playback, timed by a metronome. The pattern of paw licking observed across conditions was similar to that observed for tongue protrusions, but paw licking data are not presented here. Negative behaviors typically showed very short bout lengths, and so were scored by recording the absolute number of occurrences, as with licking responses. Negative behaviors included aversive licks (similar to those observed in Experiment 1, with narrowed tongue and very short extension), headshaking (rare), face-wiping, and gaping. Although gapes, face-wiping, and aversive licks showed comparable patterns in the various training conditions, their frequencies were low, and analyses of these individual behaviors yielded inconsistent patterns of statistical significance. Thus, we combined these three subclasses of aversive responses for statistical analysis.

Results

Training responding

Initially, only the sucrose reinforcer elicited TR or other consummatory responses. No TR or licking responses were observed during the CS-alone period on the first presentation of either CS, whereas the mean levels of positive TR and short lick responses during the post-tone (sucrose) periods on those trials were 4.71 and 4.44 responses/10-s interval, respectively. Anticipatory positive TRs and short licks were acquired over the course of training. Table 2 shows responding in the first and last training sessions. As anticipated, by the end of training, significantly more positive long-lick responses were observed during the minimally-trained tone CS than during the extensively-trained CS. By contrast, more passive short-lick responses were observed in response to the extensively-trained CS. Responses observed after sucrose presentation itself were equivalent in the two trial types. Essentially no negative TR responses were observed in training.

Table 2.

Responding during training in Experiment 2

	Group-trial type
	Maintain-Min	Maintain-Ext	Devalue-Min	Devalue-Ext
Positive responses
First training session
Pre-tone	0.75 ± 0.27	0.69 ± 0.33	0.56 ± 0.16	0.81 ± 0.41
Tone alone	0.75 ± 0.32	1.56 ± 0.19	0.81 ± 0.06	2.00 ± 0.32
Tone + sucrose	1.81 ± 0.21	2.38 ± 0.39	1.88 ± 0.22	2.94 ± 0.37
Post-sucrose	5.25 ± 0.81	5.38 ± 0.53	4.94 ± 0.34	5.06 ± 0.43
Last training session
Pre-tone	0.81 ± 0.37	0.75 ± 0.37	1.19 ± 0.26	1.19 ± 0.62
Tone alone	3.13 ± 0.94	1.25 ± 0.27	3.38 ± 0.46	1.69 ± 0.39
Tone + sucrose	3.63 ± 1.22	1.56 ± 0.16	4.38 ± 0.33	2.50 ± 0.48
Post-sucrose	6.13 ± 1.09	5.63 ± 0.39	5.81 ± 0.44	5.31 ± 0.48
Short licks
First training session
Pre-tone	0.69 ± 0.29	0.50 ± 0.35	0.38 ± 0.16	0.88 ± 0.26
Tone alone	0.56 ± 0.19	0.81 ± 0.21	0.44 ± 0.12	1.00 ± 0.23
Tone + sucrose	1.88 ± 0.60	3.38 ± 0.38	2.12 ± 0.66	4.00 ± 0.96
Post-sucrose	3.62 ± 0.62	4.62 ± 0.62	2.88 ± 0.74	4.76 ± 0.96
Last training session
Pre-tone	0.69 ± 0.53	0.50 ± 0.35	0.25 ± 0.18	0.94 ± 0.31
Tone alone	0.81 ± 0.12	2.69 ± 0.41	0.94 ± 0.28	2.00 ± 0.25
Tone + sucrose	3.88 ± 0.38	6.25 ± 1.05	4.88 ± 0.88	6.88 ± 1.07
Post-sucrose	4.00 ± 0.79	5.00 ± 0.79	4.50 ± 0.65	5.25 ± 0.63

Note. Entries are mean ± sem observations per 10-s observation period (see text for scoring details). Min signifies trials with the minimally-trained tone and Ext indicates trials with the extensively-trained tone. Negative responses were seldom observed (0.00 ± 0.00 to 0.13 ± 0.13 responses per interval across conditions).

Separate (devaluation) group X amount of training ANOVAs were performed for each observation interval, for each of the three response measures, for both the first and last training sessions. Relatively little responding was observed in the first session, although it is notable that for positive responses, the effect of amount of training was significant during the tone-alone period, F(1, 6) = 11.38, p = 0.015, and approached significance during tone + sucrose periods, F(1, 6) = 4.81, p = 0.071. Importantly, this behavior was more frequent in response to the “extensively-trained” CS, which by the end of that session had been paired with sucrose 14 times, then to the minimally-trained CS, which had been presented only 2 times. Similarly, short-lick responses were significantly more frequent to the extensively-trained CS during tone + sucrose periods, F(1, 6) = 17.78, p = 0.006, and marginally greater during tone-alone periods, F(1, 6) = 4.74, p = 0.072. Responding during the pre-CS, Fs < 1, and post-sucrose periods, Fs(1,6) < 3.09, ps > 0.129, was unaffected by amount of training for either measure. There were no effects of group, Fs (1, 6) < 1. 81, ps > 0.227, or group X amount of training interactions, Fs < 1, for these measures.

In the last training session, for both positive responses and short licks, the effect of amount of training was significant during both tone-alone and tone + sucrose periods, Fs(1, 6) > 8.32, ps < 0.028. Importantly, the effects of amount of training were in opposite directions for these two measures, with more short licks but fewer positive TR responses observed during the extensively-trained CS. There were no effects of the amount of training variable in the pre-CS or post-sucrose periods, Fs < 1. There were no effects of group, Fs (1, 6) < 2.29, ps > 0.182, or group X amount of training interactions, Fs(1, 6) < 1.30, ps < 0.298, for either measure in any observation interval. No effects or interactions were significant for any observation interval for negative responses, Fs < 1.

Devaluation responding

Figure 6 shows responding during the post-sucrose periods in the final devaluation session. Significantly more negative TR responses were observed in group Devalue than in group Maintain, F(1, 6) = 49.54, p < 0.001, whereas more positive responses were observed in group Maintain, F(1, 6) = 13.96, p = 0.010. Negative TR responses were also more frequent in Group Devalue in the sucrose interval (2.88 ± 0.79 vs 1.94 ± 0.30 responses), F(1, 6) = 45.34, p < 0.001. Neither positive (1.19 ± 0.34 vs 0.69 ± 0.19 responses) nor negative (0.00 vs 0.06 ± 0.06) TR responses differed between groups in the pre-sucrose intervals, Fs(1, 6) < 1.43, ps > 0.249. Short licks did not differ significantly between groups in any observation interval, Fs(1, 6) < 2.26, ps > 0.081.

Figure 6.

Taste reactivity responses to sucrose delivery in the final devaluation session of Experiment 2. Rats in the Devalue condition received lithium chloride injections immediately after each session whereas rats in the Maintain condition received those injections 6–8 hr later. Negative responses included licks with narrowed tongue with minimal extension, gapes, and face-washing. Positive responses shown here included only licks with a broadened and protruding tongue. Short licks involved an intermediate (normal) tongue breadth and extension. See Behavioral analysis (Experiment 2) for details.

Test responding

Figure 7 shows TR responding during the test sessions. As predicted, the rats in both groups displayed more of the appropriate TR responses to the minimally-trained CS than to the extensively-trained CS, in tone-alone, tone + water, and post-tone periods. As in Experiment 1, in group Maintain, the minimally-trained CS controlled more positive TRs (Figure 7A) than the extensively-trained CS, and the frequency of those positive TR responses was reduced by reinforcer devaluation. Furthermore, as in Kerfoot et al.’s (2007) study, rats in Group Devalue displayed negative TR responses (Figure 7C) to a minimally-trained CS after devaluation of the sucrose reinforcer, despite never having displayed such responses to the CS before. However, only low levels of those negative TR responses were observed in response to the extensively-trained CS in Group Devalue. Finally, as in Experiment 1, the extensively-trained CS evoked more short licks (Figure 7B) than the minimally-trained CS; that response was also sensitive to reinforcer devaluation.

Figure 7.

Taste reactivity responding to the minimally (Min) and extensively (Ext) trained conditioned tone stimuli in the test session of Experiment 2. Positive responses included licks with a broadened and protruding tongue, short licks involved a normal tongue breadth and extension, and negative responses included licks with narrowed tongue with minimal extension, gapes, and face-washing. See Behavioral analysis (Experiment 2) for or details. Behaviors are shown separately for pre-CS, tone-alone, tone+water, and post-water observation periods. The original sucrose reinforcer had been devalued by immediate pairings with lithium chloride injections in the Devalue groups, but its value was maintained in the Maintain groups.

Initial ANOVAs that included test order as a variable were conducted for each response measure. There were no significant effects or interactions with that variable, Fs < 1.73, ps > 0.20, so for each of the three TR measures we collapsed across test order for subsequent Group (Devalue or Maintain) X amount of training (minimal or extensive) ANOVAs. For positive TRs, both main effects and their interaction were significant during each of the observation periods except the pre-CS interval, Fs (1, 6) > 9.16, ps < 0.024. Post-hoc Tukey HSD tests showed that positive TR responding to the minimally-trained CS in Group Maintain was greater than in any of the other 3 conditions for each observation period, ps < 0.016, and that no other differences were significant, ps > 0.315. For short licks, both main effects were significant for each period except the pre-CS interval, Fs(1, 6) > 6.45, ps < 0.045, but their interaction was not, Fs < 1.76, ps > 0.233. Tukey HSD tests showed that short-lick responding to the extensively-trained cue in Group Maintain was greater than responding in any of the other 3 conditions, ps < 0.045, in the post-tone interval, and greater than responding to the minimally-trained cue in Group Devalue, p = 0.017, in the tone+water period. Finally, for negative responses, both main effects and their interaction were significant for the tone+water and post-tone periods, Fs(1, 6) > 7.52, ps < 0.034, and the main effect of groups and the group X amount of training interaction were significant for tone-alone responding, Fs(1, 6) > 6.56, ps < 0.043. Tukey HSD tests showed that negative TR responses were more frequent to the minimally-trained cue in Group Devalue than in any other condition, during the tone+water and post-tone periods, ps < 0.032, and more frequent than in either of the Maintain conditions during the tone alone period, ps < 0.033.

Discussion

The results of Experiment 2 confirmed and extended the results of Experiment 1. First, as in Experiment 1, positive TRs were acquired to a CS that received minimal training, but those responses were not displayed to an extensively-trained CS. Second, as in Kerfoot et al.’s (2007) study, learned TR responses were displayed to the minimally-trained auditory cue alone, prior to the infusion of the water substrate (albeit at a lower level). Third, also as in Kerfoot et al’s (2007) study, devaluation of the sucrose US resulted in the replacement of those positive TR responses by negative TR responses. This spontaneous transfer of negative TR responses to a CS after US devaluation was observed only to a minimally-trained CS. Finally, as in Experiment 1, apparently passive or non-hedonic short lick responses increased in frequency as training continued; these responses maintained their sensitivity to US devaluation even after extended training.

It is notable that the auditory CS controlled aversive TR responses after sucrose devaluation in Experiment 2 and in Kerfoot et al.’s (2007) experiment, which used oral cannula procedures, but not in Experiment 1, in which the sucrose US was delivered to a cup. Many authors have noted differences in both quantitative and qualitative aspects of taste aversion learning when flavors are delivered intraorally or via self-paced drinking (e.g., Schafe, Thiele, & Bernstein, 1998; Spray, Halsell, & Berrnstein, 2000; Yamamoto, Fresquet, & Sandner, 2002; but see Limebeer & Parker, 2006). It is possible that the intraoral presentation of fluids in this task especially encourages the display of aversive TR responses in testing. For example, intraoral delivery of fluids itself may be somewhat irritating, and thus this procedure may provide a more sensitive background on which aversive TR responses can be displayed. However, our studies provide little support for such an explanation. First, we attempted to minimize such an aversive component in Experiment 2 by using low infusion rates and by acclimating our rats to sucrose infusions prior to beginning training. Accordingly, we did not observe aversive TR responses in the training sessions. Second, we observed negative TR responses to sucrose in the final devaluation session in both experiments; thus, such responses could occur even with self-paced licking. We suspect that a major contributor to the differential display of aversive TR responses in test was that in Experiment 1, the rats in Group Devalue made only minimal contact with the unflavored water in test, and hence only minimal supports for the display of these behaviors were available. By contrast, the rats in Group Maintain in that experiment consumed the unflavored water in test, providing supports for the display of positive TR responses. Although the origin of this difference between the results of Experiments 1 and 2 remains uncertain, the major conclusions drawn from the two studies are identical.

Unlike Experiment 1, Experiment 2 did not include an unpaired control condition. However, the results of other experiments that used this apparatus and oral cannula procedures suggest that little TR responding would be observed without CS-sucrose pairings. First, as noted earlier, Kerfoot et al. (2007) found only occasional TR responses in testing of rats that had received 16 tones and 16 sucrose presentations in separate sessions before paired or unpaired devaluation treatment. Second, Sherwood and Holland (unpublished data) found little evidence of TR responses to the nonreinforced CS after extended discrimination training and paired or unpaired devaluation treatment. Thus it is unlikely that the changes in responding we observed here over the course of training were the result of nonassociative factors.

General discussion

Rats displayed more positive TR responses in response to auditory cues that had received small amounts of training (16 pairings) than to auditory cues that had been more extensively trained (112 pairings). This outcome was observed both when the rats were trained and tested with fluids delivered to a cup (Experiment 1) and when fluids were delivered intraorally (Experiment 2), although in the former case, positive responses were only observed after an unflavored water substrate was delivered. Because all rats received training with both minimally- and extensively-trained CSs, the differences in responding observed to those cues are not attributable to mere amount of exposure to the sucrose reinforcer.

The positive TR responses observed to minimally-trained CSs were not likely to have been the result of simple S-R learning. First, both the minimally- and extensively-trained CSs were paired with the same sucrose US. Thus, it is difficult to argue that the rats acquired responses of different forms because the CSs were paired with different unconditioned responses. Indeed, in the training phase, the sucrose US itself elicited as least as many positive responses after presentation of the extensively-trained CS as after the minimally-trained CS. Although it could of course be argued that S-R learning follows the bitonic acquisition function observed here, we see no a priori justification for that assertion. Second, when the sucrose was devalued by pairing it with LiCl, the positive taste-reactivity responses were replaced by negative responses, which had never previously occurred in the presence of auditory cues. Thus, the minimally-trained CSs controlled sensory-evaluative responses appropriate to the current evaluation of the sucrose reinforcer, rather than behaviors elicited by sucrose at the time of training. This finding extends the observations of Berridge and Schulkin (1989) in three ways. Whereas Berridge and Schulkin (1989) found a spontaneous shift from negative to positive TR responses when the value of a salt solution was increased by manipulating a motivational state, we showed a complementary shift from positive to negative responses when the value of sucrose was reduced by an associative manipulation. Most important, this shift was observed only after relatively small numbers of pairings of an auditory cue with sucrose.

Although access to these sensory-evaluative aspects of the sucrose reinforcer was confined to early stages of training, that limitation was not indicative of a transition to S-R or other inflexible forms of learning. Both food cup entry (Experiment 1) and short licks (Experiments 1 and 2) increased monotonically over the course of training, and retained their sensitivity to devaluation with extended training. Thus, the performance of those behaviors was also mediated by the current significance of the sucrose reinforcer, but apparently in ways that did not involve access to hedonic evaluation or TR responses.

On the surface, our observation of a nonmonotonic acquisition function for positive evaluative responses, but not for other CRs, is reminiscent of a pattern of data presented by Kaye and Pearce (1984), who found that whereas food cup responses increased monotonically with extended light-food pairings, a light-directed orienting response (OR) initially increased in frequency, but then declined as training continued. They interpreted this outcome in the context of the Pearce and Hall (1980) model. In that model, the associability of a CS (the rate with which it may enter into new associations) varies as a function of the discrepancy between expected and actual USs. As the US becomes more reliably predicted, the associability of a CS will decline. Thus, the more CS-US pairings, the more slowly new learning could be acquired to that CS. A number of studies showed both that the associability of CSs declines as training proceeds, and that such losses in associability can be restored by disrupting the consistent CS-US relation, for example by changing or removing the US (Hall & Pearce, 1979; Pearce & Hall, 1980).

An important feature of this model is that CS associability is assumed to influence only learning, and not performance; the ability of a CS to elicit CRs, once they are established, is not influenced by its associability. Thus, CSs maintain their ability to elicit previously-established CRs even as they lose the ability to acquire new learning. Kaye and Pearce (1984) asserted that in their experiments, whereas food cup responding provided a measure of a cue’s associative strength, the OR indexed its associability. The pattern of data observed in our experiments could be interpreted in the same way if positive TR responses could be construed as ORs to the auditory CSs, and the other measures as CRs that index CS-food association. However, it seems unlikely that the positive TR response is an OR to the auditory CSs we used, or that it otherwise serves as an index of associability rather than of strength of association. Unlike Kaye and Pearce’s (1984) ORs, this response was not observed to the CSs prior to conditioning, and it did not reemerge in extinction testing, when according to the Pearce-Hall (1980) model, the CS’s associability would again increase. Furthermore, it is unclear how the replacement of positive TR responses by negative responses after devaluation in Experiment 2 could be interpreted within this framework.

The differential sensitivity of evaluative and other learned responses to the amount of training observed here may be more profitably compared to Holland’s (1998, 2005) observations of the loss of a cue’s ability to participate in mediated learning but continued sensitivity to US devaluation with extended training. Using the same training procedures as were used in Experiment 1 (except for the use of a pellet rather than a liquid sucrose US), Holland (2005) found that only the minimally-trained CS could mediate the conditioning of an aversion to the food US after CS-LiCl pairings, but at the same time, in other, identically-trained rats, food cup responding was equally sensitive to US devaluation by food-LiCl pairings after either minimal or extended training. However, Holland (2005) found that the loss with extended training of the CS’s ability to establish mediated learning was not ameliorated by the induction of surprise by using partial reinforcement contingencies or by introducing a block of extinction trials prior to CS-LiCl pairings. Thus, as with the present data, an account for Holland’s (1998, 2005) findings based on the Pearce-Hall (1980) model was not supported.

Holland (1990, 1998, 2005) asserted that in the initial stages of acquisition, CSs gained access to early-stage, pre-evaluative processing of the US, for example, activating perceptual systems, but as training progressed, that access was replaced by access to a representation that was dominated by more conceptual or motivational aspects of US processing. In the most radical version of this view (Holland, 1990), early in training the CS activates perceptual processing normally activated by the food itself, including “tasting” of the absent food US reinforcer. Within this perspective, any function normally controlled by the taste of the US should be accessible by the CS. Thus, mediated learning of an aversion to that reinforcer would be expected if the CS was paired with LiCl injection, because the rat would experience illness after tasting the absent US. Similarly, that CS should control evaluative TR responses appropriate to that taste, as observed here. Because the CS would activate “pre-evaluative” perceptual processing of the food US, the TR observed would be consistent with the current evaluation of the taste of that food, and thus would be sensitive to post-training changes in reinforcer value, as observed here.

Within this view, as training continues, the CS loses access to this low-level perceptual processing of the US, and instead activates aspects of reinforcer processing that do not include perceptual experience itself or the ability to control evaluative responses appropriate to the current evaluation of that perceptual experience. These more conceptual aspects of the US nevertheless must include both sensory and some sort of motivational information, because reinforcer devaluation effects remain highly US-specific even after the CSs lose their ability to control TR responses or mediate new taste aversion learning. For example, Holland (1998) examined both mediated learning and reinforcer devaluation effects after training rats with two auditory CSs and two food USs. Pairings of one of those CSs with LiCl produced a selective aversion to its food partner after minimal but not extended training. Other rats that received pairings of that food itself with LiCl continued to display selective devaluation when tested with the two CSs even after extended CS-US training. Thus, even after extended training, CSs can activate reinforcer representations that include rich sensory information. Nevertheless, the representations activated by CSs after minimal and extended training can be distinguished by their ability to participate in new learning and by their ability to control appropriate evaluative TR responses. This distinction resembles one suggested by Konorski (1967, e.g., pp. 170–181), who distinguished between the associative activation of an “hallucination” and of an “image” by a CS. In the former case, the CS was thought to evoke activity of sensory “projective” units, which could resemble the activity of those units when they were activated by the US itself. By contrast, in the latter case, the CS activated only “gnostic” units which coded information about the US, but which did not normally generate perceptual processing. Casually speaking, an hallucination includes sensory experience of an absent object whereas one can imagine that object and its sensory properties without experiencing them.

Additional evidence for associative control of perceptual taste processing by auditory CSs was provided by Holland (1990, Exp. 2 and 3). Water-deprived rats first received pairings of tone and noise CSs with sucrose or saline solutions. Next, in their home cages the rats were trained with either positive or negative patterning taste aversion discrimination procedures. In the positive patterning procedure, a sucrose + saline compound was paired with LiCl and each individual flavor was nonreinforced, whereas in the negative patterning procedure, the compound was nonreinforced, but each flavor alone was paired with LiCl. Finally, rats were allowed to drink plain water in the experimental chambers, while presentations of the tone, noise, and a noise + tone compound were superimposed. When either CS was presented alone, the rats displayed TR responses appropriate to the current value of the individual sucrose and saline solutions, that is, positive TRs in the rats that had received positive patterning and negative TRs in the rats that had received negative patterning. Critically, when the noise and tone were presented simultaneously for the first time, the rats displayed the opposing TR responses, those appropriate to the sucrose + saline compound (that is, negative TRs in positive patterning and positive TRs in negative patterning). It is difficult to interpret this pattern of results without recourse to some control of taste processing by the auditory CSs alone.

Although a framework based on the associative control of perceptual processing encompasses Holland’s (1990, Exp. 2 and 3) data and Holland’s (1998) findings in mediated learning, the data presented in the present article themselves demand only that minimally-trained CSs activate systems that control evaluative TR responses and that such access declines with extended training. Notably, those TR responses are not typically thought to directly index taste perception, but rather the results of some computation of the palatability of a taste (e.g., Berridge & Schulkin, 1989). Thus, whereas the evaluation of the taste of sucrose changes as sucrose-illness associations are acquired (e.g., Berridge et al., 1981; Garcia, et al., 1985; Pelchat et al., 1983), that taste itself may remain constant. Observations that CSs often acquire the motivational significance of their USs are consistent with a range of motivationally-oriented learning theories (e.g., Berridge, 2001; Bindra, 1978; Toates, 1986). More critically, observations from devaluation experiments that a CS often controls responding that is appropriate to the current evaluation of the US rather than to its evaluation when it was originally paired with the CS, show that such a CS activates a representation of the US that can be updated by subsequent experience. In particular, CSs must be associated with some aspect of US processing that precedes computation of its motivational value. Finally, indications that under different training conditions, CSs control different updatable aspects of US processing (as in the present studies) indicate that a range of psychological and brain functions may come under the control of associative learning. Experiments such as those reported here provide a first step in exploring the conditions for, and nature of, those learning functions.

Evidence for changes in the nature of learning over the course of training also comes from studies that show shifts in the brain systems engaged as learning proceeds. For example, the results of a series of investigations by Packard and his colleagues (e.g. Packard & McGaugh, 1996; Poldrack & Packard, 2003) indicate that the behavior of rats in plus mazes is controlled primarily by hippocampus-dependent learning of place-food relations early in training, but becomes more controlled by caudate-dependent stimulus-response learning as training progresses. Similarly, Bussey et al. (1996) attributed initial stimulus-reinforcer learning in an operant conditional discrimination task to anterior cingulate systems, and later stimulus-response learning to posterior cingulate systems. Typically, these investigators have interpreted their findings as representing shifts from broader, more flexible learning early in training to more focused and automatic learning later in training.

The results reported here differ from those just described in that even learning under conditions of “extended” training was still flexible, in that it remained susceptible to post-training reinforcer devaluation effects. Different response systems use different neural systems, which may display plasticity with different characteristics. After assessing TR responses to a minimally-trained tone CS, in the absence of sucrose, Kerfoot et al. (2007) sacrificed their rats to examine the expression of FOS, the protein product of the activity-dependent immediate-early gene c-fos. They found learning-dependent FOS expression in a number of brain regions known to be critical for learning that is sensitive to outcome devaluation (basolateral amygdala and orbitofrontal cortex; Gallagher et al., 1999; Gottfried et al., 2003; Hatfield et al., 1996), in regions related to the display of TR responses (accumbens shell; Reynolds & Berridge, 2002), and in regions related to processing of taste information (gustatory cortex; Kiefer & Orr, 1992). It would be intriguing to determine if a more extensively-trained CS would fail to induce FOS in the accumbens shell or gustatory cortex.