Differential effects of serotonin-specific and excitotoxic lesions of OFC on conditioned reinforcer devaluation and extinction in rats

Abstract

The orbitofrontal cortex (OFC) is critical for behavioral adaptation in response to changes in reward value. Here we investigated, in rats, the role of OFC and, specifically, serotonergic neurotransmission within OFC in a reinforcer devaluation task (which measures behavioral flexibility). This task used two visual cues, each predicting one of two foods, with the spatial position (left-right) of the cues above two levers pseudorandomized across trials. An instrumental action (lever press) was required for reinforcer delivery. After training, rats received either excitotoxic OFC lesions made by NMDA (N-methyl-D-aspartic acid), serotonin-specific OFC lesions made by 5,7-DHT (5,7-dihydroxytryptamine), or sham lesions. In sham-lesioned rats, devaluation of one food (by feeding to satiety) significantly decreased responding to the cue associated with that food, when both cues were presented simultaneously during extinction. Both types of OFC lesions disrupted the devaluation effect. In contrast, extinction learning was not affected by serotonin-specific lesions and was only mildly retarded in rats with excitotoxic lesions. Thus, serotonin within OFC is necessary for appropriately adjusting behavior towards cues that predict reward but not for reducing responses in the absence of reward. Our results are the first to demonstrate that serotonin in OFC is necessary for reinforcer devaluation, but not extinction.

The orbitofrontal cortex (OFC) is critical for behavioral flexibility, including maintaining reward expectancies (e.g. [1]) One measure of behavioral flexibility is conditioned reinforcer devaluation. In these tasks, subjects associate cues with primary reinforcers (e.g. foods), with each cue predicting a discrete food. One food is then “devalued” by selective satiation (feeding to satiety) or taste aversion (pairing with illness). Following devaluation, normal subjects shift behavioral responses away from the cue associated with the devalued food, even without an opportunity to experience the cue in the presence of the devalued food (e.g., under extinction).

Other markers of behavioral flexibility include extinction learning (the ability to suppress responding when a response is no longer reinforced) and reversal learning (the ability to suppress responding to a previously reinforced stimulus in favor of responding to a previously non-reinforced stimulus, when the reward contingency is reversed). Disruption of OFC function either by lesion or transient inactivation in both primates [2–4] and rodents [5, 6] impair reinforcer devaluation, extinction [7, 8], and reversal learning [2, 9].

A major neuromodulator that has been implicated in many of these behaviors is serotonin [10–12]. Serotonin dysfunction has been associated with a variety of neuropsychiatric disorders that involve altered behavioral flexibility, including obsessive-compulsive disorder [13]. Selective depletion of orbitofrontal serotonin disrupted reversal learning in monkeys and rats [14–16]; in contrast, depletion of orbitofrontal serotonin did not disrupt extinction learning in monkeys [17]. The effect of orbitofrontal serotonin depletion on conditioned reinforcer devaluation has not been tested in any species. Given the role of orbitofrontal serotonin in other tasks (previous findings above), we expected serotonin depletion to impair reinforcer devaluation, but not extinction learning in the present task.

Male Long-Evans rats (N=37; Charles River, Frederick, MD) were maintained at the Georgetown University Medical Center; housing conditions and training procedures were the same as those described previously [18, 19]. The study was conducted approval from the Georgetown University Animal Care and Use Committee and in accordance with the Guide for Care and Use of Laboratory Animals.

Briefly, standard rat operant chambers containing two levers, with cue lights positioned above each, were used. Both levers were present throughout training and testing. Cue1 was a green light flashing at a frequency of 1Hz and Cue2 a red light flashing at a frequency of 5Hz. Responses to Cue1 resulted in the delivery of a sugar pellet and responses to Cue2 a chocolate pellet. The association between the cues and specific rewards was constant but the position of each cue above the left versus the right lever was pseudorandomized across trials.

Rats were trained through progressive stages to press a lever with either Cue1 or Cue2 flashing above it in order to receive a food pellet. On any given trial one lever was active (light above it illuminated) and one was inactive (light not illuminated). Responses on the active lever resulted in the pellet delivery. During the final stage of training, each rat was allowed 30s to respond on the active lever with the pellet delivered on a variable-ratio-nine (VR9) schedule in a 20-minute session. Criterion level of performance was set at 85% correct over three consecutive days. When the rats reached criterion, they were assigned to one of three surgical groups, each with equivalent pre-surgical performance: 1) bilateral excitotoxic lesions of OFC, 2) serotonin-specific lesions of OFC, 3) sham lesions.

Surgery was done under anesthesia (equithesin, 2.5ml/kg, i.p.) and aseptic conditions (surgical details in [18]). Excitotoxic lesions were made by intracerebral infusions of NMDA (N-methyl-D-aspartic acid, Sigma, St. Louis, MO; 12.5 μg/μl, dissolved in saline), following the procedure described by Ostlund and Balleine [20]. Serotonin-specific lesions were made by intracerebral infusions of 5,7-DHT (5,7-dihyroxytryptamine 20ug/ul dissolved in saline). The latter group received an injection of desipramine (25 mg/kg, ip) to prevent damage to norepinephrine neurons. NMDA (0.4 μl) or 5,7- DHT (0.4 μl) were infused at the coordinates: 3.5 mm anterior to bregma, 3.2 mm lateral to midline, and 4.7mm ventral to the surface of the skull. Sham animals received identical procedures but were infused with saline instead. Following surgery, rats received analgesics and subcutaneous saline. After the completion of behavioral testing (as described below), animals were euthanized and lesions confirmed histologically (Figure 1 and Supplementary Methods and Results). Twelve animals with excitotoxic lesions of OFC, 9 animals with serotonin specific lesions of OFC, and 12 sham-lesioned animals had lesions as intended and were, thus, included in data analyses.

Figure 1.

(A) All cases of excitotoxic lesions are superimposed on drawings of coronal sections: the darker the area, the more overlap across lesions. The numbers represent distance in millimeters from bregma and approximately correspond to the equivalent planes of the atlas of Paxinos and Watson (2010). (B) Representative low power photomicrographs of Nissl stained section showing the outline of frontal areas. The intended lesion is indicated as VO/LO (ventrolateral and lateral orbital regions) and is located dorsal to the rhinal fissure, ventral to corpus callosum and medial to insula. (C) Photomicrograph of Nissl stained tissue from sham and excitotoxic (NMDA) lesioned animals. The square in C1 and C3 indicate the approximate area shown at higher power in C2 and C4, respectively. (D) Maps of serotonin-specific lesions constructed as described in (A). (E) Photomicrographs documenting serotonin immunohistochemistry (E1 and E2) and Nissl staining (E3 and E4) in sham and 5,7-DHT treated animals, respectively. CC- corpus callosum, PrL-Prelimbic, IL- infralimbic, SSC, somatosensory cortex, IC- insular cortex, rf = rhinal fissure.

Postoperatively, rats were retrained to criterion prior to selective satiation and testing in instrumental or consummatory probes. For selective satiation, rats were presented with 25g of one of the two foods (e.g. chocolate pellets), and monitored until they stopped eating. There was no significant difference in food consumption during selective satiation between groups as shown by a one-way ANOVA (F_2,31=0.56, p=0.58). This result indicated that OFC lesions did not affect the amount of food the rats consumed during satiation.

We then assessed whether devaluation of a food reinforcer was transferred to the associated cue during an instrumental probe session. Successful transfer would manifest as fewer lever presses to the cue associated with the devalued reinforcer (Cue_D) than the cue associated with the non-devalued reinforcer (Cue_ND). To prevent rats from re-learning the value of cue by pairing with devalued food reinforcer, the probe was conducted under extinction (i.e., no pellets were delivered).

In the instrumental probe, the cues above both levers were illuminated simultaneously. Cue1 and Cue2 were counterbalanced across the lever positions. Rats were allowed 15 s to respond in each trial of the 5-minute session (S1). They were retrained to criterion and, at least 2 days later, tested in a second instrumental probe session (S2) following devaluation of the alternative reinforcer. To rule out the possibility that the lesions affected the overall rate of responding during the instrumental probes, we assessed the number of responses each rat made to both cues (Cue1 + Cue2) in each session and compared it between groups. A repeated-measures ANOVA with lesion as a between-subject factor and session as a within-subject factor yielded no significant effect of lesion (F_2,64=0.93; p=0.40), session (F_1,64=0.10; p=0.75), or interaction of the two factors (F_2,32=0.80; p=0.46), indicating that neither lesion affected the overall rate of responding (i.e. the number of lever presses).

To determine if there was an effect of the lesions on reinforcer devaluation, we transformed the data for each rat into a devaluation index as previously described (West et al., 2012). Although there were no significant group differences in the number of lever presses, there was a large variability in this measure across rats within each group and the devaluation index allowed us to overcome this variability. It was calculated according to the formula: (Responses to Cue_ND minus responses to Cue_D) / (Total responses). Thus, if a rat responded equally to both cues, the devaluation index would be 0. If a rat responded only to Cue_ND or only to Cue_D, the devaluation index would be 1 or −1, respectively. According to the formula, if a rat showed a devaluation effect, its devaluation index was positive.

Aligned rank transformed ANOVA for nonparametric data (26) with lesion as a between-subject factor and session (S1, S2) as a within-subject factor, applied to the devaluation indices, yielded no significant effect of session (F_1,32=0.17; p=0.68) or session by lesion interaction (F_2,32=0.23; p=0.80). Therefore, we collapsed the data across the two probe sessions, as we have done previously (18). Applying the same analysis to the collapsed data, we found a significant effect of lesion (F_2,32=4.23; p<0.05). Post-hoc Mann-Whitney (planned comparisons, p<0.05, 1-tailed) tests showed that both lesion groups had lower devaluation indices than the sham-lesioned group.

In addition, for each group, devaluation indices were analyzed using a one-sample t-test comparing the scores to 0. Scores that were significantly higher than 0 indicated that the group showed a devaluation effect. The mean devaluation index for sham rats was a positive value, significantly different from 0 (t=2.7, p<0.05; Figure 2). In contrast, the mean devaluation indices for rats with both OFC lesion types were not significantly different from 0 (t=0.13, p=0.1; t=0.7, p=0.25, for excitotoxic and serotonin-specific lesions, respectively), indicating that rats responded equally to both cues.

Figure 2.

Instrumental probe of reinforcer devaluation. Bars indicate the mean (+SEM) devaluation index for sham, excitotoxic and serotonin-specific lesion groups Devaluation index significantly different from zero; * = p<0.05.

The consummatory probe was used to determine if selective satiation resulted in successful devaluation of the food, manifested by consuming less of the devalued food (Food_D; e.g. chocolate pellets) as compared with the non-devalued (Food_ND; sugar pellets) when the foods were freely available. In the consummatory probe each rat was given access (30 minutes) to two dishes, one filled with the satiated food (e.g. chocolate pellets) and the other with the non-satiated food (sugar pellets) immediately following selective satiation. At least 2 days later, rats were satiated with the alternative food. The total amount of devalued (satiated) food pellets (Food_D; grams) consumed was then summed for each rat across the two sessions and compared with the total amount of non-devalued food pellets (Food_ND) consumed across the two sessions. The devaluation index was calculated according to the formula: (Total amount of Food_ND consumed minus total amount of Food_D consumed) / (Total amount consumed).

The mean devaluation indices for all groups were positive values, significantly different from 0, as shown by one-sample t-tests (t=9.2, t=8.0, t=7.8, p <0.05; Figure 3), indicating that neither type of the OFC lesions resulted in an impairment of consummatory behavior.

Figure 3.

Consummatory probe of reinforcer devaluation. Bars indicate the mean (+SEM) devaluation index for sham, excitotoxic and serotonin-specific lesion groups. Devaluation index significantly different from zero * = p<0.05.

We next examined potential differences in extinction of responding during the two instrumental probes. Each trial was terminated if the rat completed the number of responses required by the VR9 schedule or by the maximum duration of 15 s elapsing. The duration of the probe session was limited to 5 minutes and the maximum number of trials recorded during this time period for any animal was 21; thus, responses were cumulated into three, seven-trial bins (trials 1–7, 8–14, and 15–21).

The total number of responses (Cue_D plus Cue_ND) for each of the three bins was recorded separately for the first (S1) and the second (S2) devaluation. Data were analyzed by within-subject ANOVA with repeated measures for the bins. As shown in Figure 4A, all groups decreased responding across bins in S1, demonstrating extinction learning. These effects were revealed by ANOVA with repeated measures that showed a significant main effect of bin (F_2,62=32.24; p<0.05), but no significant main effect of lesion status (sham, excitotoxic, serotonin, F_2,31=2.16; p=0.13), or lesion-status-by-bin interaction (F_4,62=1.57; p=0.19). Similarly, all groups showed a significant decrease in responding across bins in S2 (Figure 4B), with a significant effect of bin (F_2,62=24.51; p<0.05), but neither a significant effect of lesion nor a lesion-status-by-bin interaction (F_2,31=0.41; F_4,62=1.06; ps>0.05). This demonstrates that all three groups showed extinction learning. Even though the overall ANOVA was not significant, animals with excitotoxic OFC lesions were mildly but significantly retarded in extinction of responding in the first session (S1), as shown by the number of responses in Bin 2 that was higher than those of the other two groups (p<0.03; Bonferroni-Holm step down planned comparison; Figure 4A). Given the previous findings of impairment in extinction after OFC lesions (7, 8), this finding is not completely unexpected and merits further investigation.

Figure 4.

Extinction learning. Session 1 of testing (A), Session 2 of testing (B). Each data point indicates the mean (+SEM) number of lever presses per bin (bin=7 trials) on the instrumental probe of reinforcer devaluation. Text in the lower left of each panel shows the p values for the effects in the ANOVA. * represents difference from both sham and serotonin lesion (Bonferoni-Holm planned comparisons).

Here we have shown that excitotoxic OFC lesions disrupted performance on the reinforcer devaluation task and that selective damage to serotonin fibers was sufficient to yield a comparable impairment. We also found that OFC lesions did not significantly affect extinction, except for a mild retardation in the group with excitotoxic lesions.

The reinforcer devaluation task used in the present study includes both pavlovian and instrumental components. It is, thus, similar to the task used in monkeys, in which each of two sets of visual cues (objects) predicts one of two rewards and an instrumental action (moving an object) is required to obtain the reward [3, 19]. Consistent with the similarity of these tasks, the impairment we found in rats with OFC lesions is akin to that seen after lesions or transient inactivation of OFC in monkeys [2, 3]. Our findings are also consistent with the impairment seen following OFC lesions in rats [5, 6] in studies that employed pavlovian tasks.

In contrast to pavlovian tasks, OFC lesions have been previously reported to have no effect on performance in an instrumental task, in which no cues other than the spatial position of the levers signaled the differential outcome leading to the conclusion that OFC is not critical for instrumental outcome devaluation [20]. Although in our task an instrumental action (pressing a lever) was required to receive the reinforcer, the predictive information about the specific reinforcer to be delivered was provided by two distinct visual cues. Our data are compatible with the suggestion [21] that the presence of cues that signal outcomes, regardless of whether an instrumental action is required, engages OFC.

Here we have shown for the first time that serotonin within OFC is critical for normal conditioned reinforcer devaluation. These data complement prior findings that depletion of orbitofrontal serotonin disrupts reversal learning in both monkeys and rats [15, 16]. In contrast to reversal learning, serotonin depletion did not impair extinction learning in our rats, which is consistent with a previous finding in monkeys [17].

The role of serotonin in behavioral flexibility has been also previously examined using systemic pharmacological manipulations, as well as genetic techniques. Systemic pharmacological treatments that increase serotonin (e.g., administration of the serotonin-selective reuptake inhibitor, fluoxetine) result in improved reversal learning [22], without affecting extinction [23]. Using genetic approaches, several laboratories have undertaken a series of studies in a serotonin transporter knockout rats and mice which support a role for serotonin in behavioral flexibility [24]. Behaviorally, these animals displayed improved performance on prefrontal dependent tasks including reversal learning [22]. Paradoxically, the knockout rats showed impaired reinforcer devaluation [25], a phenotype that our data suggest is consistent with reduced serotonergic function in OFC. This difference may be due to an inverted U type dose-response function for serotonin signaling on reinforcer devaluation (i.e., both supra and suboptimal levels of serotoninergic signaling disrupt behavioral performance). Experiments assessing the effects of acute elevation of serotonin in OFC would complement our present data.

Performance on the conditioned reinforcer devaluation task does not require new learning, but instead requires the animals to integrate previously learned associations with new information about the devalued reinforcer without ever experiencing the cues with the devalued reinforcer. In contrast, during extinction, rats are required to learn to suppress responding to a previously rewarded instrumental action or cue. Perhaps this difference accounts for the selective vulnerability to orbital damage we have described.

In summary, we have shown that rats with excitotoxic lesions of OFC are profoundly impaired at conditioned reinforcer devaluation and mildly retarded (if at all) at extinction learning. Rats with OFC depletion of serotonin showed a comparable impairment in devaluation with no effect on extinction. These findings are consistent with previous work [14–17] and add to a growing literature implicating OFC, and serotonin within OFC, in behavioral flexibility.

Highlights.

• Orbitofrontal cortex (OFC) excitotoxic and serotonin specific lesions disrupt reinforcer

• devaluation in a task with cued and instrumental components

• Serotonin specific lesions leave extinction learning intact

• Rats with OFC excitotoxic lesions show a mild retardation of extinction learning