Effect of a 5-HT7 Receptor Antagonist on Reversal Learning in the Rat Attentional Set-Shifting Test

Abstract

5-HT7 receptor antagonism has been shown to ameliorate ketamine-induced schizophrenia-like deficits in extradimensional set-shifting using the attentional set-shifting task (ASST). However, this rodent paradigm distinguishes between several types of cognitive rigidity associated with neuropsychiatric conditions. The goal of this study was to test 5-HT7 receptor involvement in the reversal learning component of the ASST because this ability depends primarily on the orbito-frontal cortex, which shows strong 5-HT7 receptor expression. We found that impaired performance on the ASST induced by NMDA receptor blockade (MK-801, 0.2 mg/kg) in 14 rats was reversed by coadministration of the 5-HT7 receptor antagonist SB-269970.The strongest effect was found on the reversal phases of ASST, whereas injection of SB-269970 alone had no effect. These results indicate that 5-HT7 receptor mechanisms may have a specific contribution to the complex cognitive deficits, increasing perseverative responding, in psychiatric diseases, including schizophrenia, depression, and anorexia nervosa, which express different forms of cognitive inflexibility.

Graphical Abstract

INTRODUCTION

The serotonin 7 (5-HT7) receptor is implicated in the pathogenesis and treatment of psychiatric diseases, including schizophrenia, depression, and sleep disorders. Several antipsychotics¹ (most prominently lurasidone² and amisulpride³) and antidepressants (e.g., vortioxetine⁴) include 5-HT7 in their rich receptor-binding profile. Clozapine, for example, has a higher affinity to 5-HT7 receptors than to D2 receptors.⁵ Cortical 5-HT7 receptor abnormalities were reported in human subjects with schizophrenia,^6,7 and the roles of 5-HT7 mechanisms were shown in rodent models of schizophrenia.^3,8–10 The latter included not only the antipsychotic-like efficacy of the selective 5-HT7 receptor antagonist SB-269970, attenuating amphetamine and ketamine-induced hyperactivity¹¹ and reversing prepulse inhibition deficits,¹² but also its procognitive effect in acute^3,13,14 and subchronic models.^9,10,15 Even though data regarding the antidepressant- or antipsychotic-like effects of these compounds remain mixed (see, e.g., ref 16), there is growing research interest in studying the therapeutic potential of 5-HT7 receptor mechanisms to manage cognitive dysfunction in schizophrenia and depression.

Preclinical evidence indicates that second generation antipsychotics may potentially reduce cognitive impairments according to their unique receptor-binding profiles, including the 5-HT7 receptor. Although not selective for 5-HT7 receptors, drugs with high 5-HT7 receptor affinity were tested in clinical trials and in a range of cognitive studies in rodents, including novel object recognition, spatial memory, executive function, and cognitive flexibility, pursuing lurasidone^{2,13,14,17,18} and amisulpride^3,19,20 in schizophrenia and vortioxetine^21–24 in depression. Observations indicative of these drugs’ procognitive potential keep accumulating. Importantly, it was found that their effects, on the one hand, were reversed with coadministration of the 5-HT7 receptor agonist AS19^13,25,26 and, on the other hand, could be replicated by the selective 5-HT7 receptor antagonist SB-269970.^3,26

Cognitive deficits extend to a wide range of cognitive functions, including executive functions, which is one of the cognitive domains least responsive to antipsychotic drugs.^10,27 Cognitive flexibility, an important component of executive functioning, has different forms impaired in schizophrenia. Amelioration of schizophrenia-relevant deficits of cognitive flexibility was tested using different rodent models. The effect of the 5-HT7 receptor antagonist SB-269970 specific to reversal learning was reported in the subchronic^9,10 and that of attentional set-shifting deficits in the acute³ N-methyl-d-aspartatic acid (NMDA) receptor hypofunction model. However, the two models, i.e., acute and subchronic, were tested using different behavioral paradigms. Whereas reversal learning was tested in an operant task after subchronic administration of phencyclidine (daily injections for several days), in both rats and mice, set-shifting was studied using the attentional set-shifting task (ASST) after acute administration of ketamine in rats. Although ASST allows for the identification of several types of cognitive rigidity, reversal learning did not appear to be sensitive to ketamine,³ thus hindering the examination of 5-HT7 receptor action. To resolve this apparent discrepancy, in this study, we tested the potential effect of 5-HT7 receptor blockade in ASST using the longer lasting noncompetitive NMDA receptor antagonist, MK-801, which was shown previously to disrupt reversal learning in different spatial and nonspatial tasks under reversal conditions.^28–32

The commonly used rodent ASST allows for the identification of several types of cognitive rigidity, which involve different prefrontal cortex (PFC) networks and may be affected by different receptor systems. Set-shifting and reversal learning were shown in the ASST to primarily depend on medial PFC and orbitofrontal cortex networks, respectively.^33–37 Since 5-HT7 receptors show stronger orbitofrontal expression compared with medial PFC,³⁸ we hypothesized that the 5-HT7 receptor antagonist SB-269970 may avert or diminish the deficit in reversal learning induced by MK-801.

RESULTS AND DISCUSSION

Reversal learning was tested with the widely used and standardized ASST developed by Birrell and Brown.³⁷ In this paradigm, the rat has to go through a series of seven phases in which it has to find food in one of two pots. The pots are scented with different odors and are filled with different digging materials, under which the food is hidden. The seven phases, delivered in a sequence, one after the other, differ in terms of which odor or digging medium is set as the cue signaling the baited pot. At each stage, the rat has to learn which cue to use to find the food, and once the learning is complete (six consecutive correct trials), the task changes and the rat has to change strategy accordingly. After the initial stages of a simple (SD) and a compound discrimination (CD), in which the two dimensions are introduced, the rat’s effort is measured to make the right choice when clues change: first within the same dimension (ID) and next by switching between dimensions (ED). Each of these tests are followed by reversals (Rev1, −2, and −3 after CD, ID, and ED, respectively), in which the same clues and dimensions are used as in the previous trial, except that the +/− reward assignment is switched.

The rats were divided into two groups and tested according to one of two experimental designs. Rats in Group 1 (n = 8) were tested on ASST on three occasions (1 control and 2 drug injections), whereas Group 2 rats (n = 6) were tested six times after multiple injections (see below). This design was based on two considerations. First, control testing was necessarily performed first in all rats to generate reliable control baseline. As testing proceeded, however, the rats may have learned and improved on the task. To compensate for a potential shift in baseline performance, the test results of the control ASST (after saline injection) at the beginning of the experimental series was used in group 1, whereas in Group 2 the results of the control ASST was repeated at the end of the experiment and were used for statistical analysis. Second, as shown in prior studies,^39,40 injection of MK-801 elicited continuous waking, hyperactivity, and stereotypic movements which lasted for 4–5 h. Due to behavioral impairments induced by MK-801, the rats did not take food for the first ~2 h after injection, even if food was scattered in the recording arena. Therefore, ASST started only when the rats were able to find the food, hold it, and eat it. This design was comparable to a previous study in which the ASST started 45 min after the shorter acting ketamine injection.³ Instead of ketamine, we used the more selective NMDA receptor antagonist MK-801, which has a longer effect and is thus more suitable for behavioral paradigms lasting for several hours. Acute abnormal network activity induced by MK-801 (enhanced gamma activity) was shown to last 4–5 h,^39,40 i.e., covering the time of the ASST starting approximately 2 h after injection. For comparison, abnormal behavior and enhanced cortical gamma activity last less than 1 h after ketamine.

In the control ASST, the rats required more trials to learn reversals, especially Rev1 (see Methods), than for other discriminations. Additionally, the most severe impairments after NMDA receptor blockade were detected in the reversal stages. This latter effect was completely eliminated by coadministration of the 5-HT7 receptor antagonist, SB-269970 (Figure 1). Two-way ANOVA confirmed the effect of the main factors (ASST-discrimination: F(6,259) = 15.05, p = <0.0001; and drug: F(2,259) = 19.75, p = <0.0001; Groups 1 and 2 combined, n = 14) but revealed no significant interaction effect (F(12,259) = 9.51, p = 0.244). Posthoc Bonferroni comparison showed significant differences at the p = 0.05 level between control and MK-801 and between MK-801 and MK-801+SB-269970 injections but no difference between the latter and control ASST. The same results were found in two-way ANOVA after the three reversals (Rev1, −2, and −3) were combined (ASST-discrimination: F(4,259) = 20.99, p = <0.0001, drug: F(2,259) = 11.78, p = <0.0001, interaction: F(8,259) = 0.89, p = 0.529). Posthoc Bonferroni comparison also showed significant differences between MK-801 and the other two conditions. On average, trials to criterion on the reversals increased by 25% after MK-801 (13.1 ± 0.8) compared with control (9.9 ± 0.5) and MK-801+SB-269970 coadministration (10.4 ± 0.4) (Figure 2A). This pattern was consistently recorded in individual experiments (in 11 of the 14 rats; Figure 2B).

Figure 1.

Effect of the 5-HT7 receptor antagonist SB-269970 on cognitive deficits induced by NMDA receptor blockade, measured using the ASST. Results represent average number of trials to reach the criterion of six consecutive correct trials, where higher means indicate more impairment (Groups 1 and 2 combined, N = 14). SB-269970 (2 mg/kg) was injected immediately before MK-801 (0.2 mg/kg) administration. ASST started about 2 h after injection when the rats could find, hold, and eat food scattered in the arena. Symbols: ^#p < 0.1, ^##p < 0.05, ^###p < 0.01 saline vs MK-801; *p < 0.1, **p < 0.05, ***p < 0.01 MK-801 vs SB-269970+MK-801.

Figure 2.

Amelioration of the reversal learning deficit (Rev1, −2, and −3, combined) induced by NMDA receptor blockade by SB-269970, as measured using the ASST. (A) Group averages (N = 14); (B) individual experiments (dashed lines: experiments in which the pattern differed from group). Results represent trials to reach criterion averaged over all three reversal learning phases of ASST. A higher number of trials indicates more impairment in reversal learning ability.

In six rats (Group 2), the protocol also included additional ASST drug trials, one testing SB-269970 alone and one in which SB-269970 was injected approximately 2 h after MK-801, i.e., immediately before starting the ASST. Two-way ANOVA showed significant differences in population means for ASST-discrimination phase (F(6,192) = 5.14, p = <0.0001) and drug injection (F(4,192) = 9.12, p = <0.0001) with no significant interaction (F(24,192) = 0.75, p = 0.79). Posthoc Bonferroni comparison revealed significant differences between performances after MK-801 injection alone vs all other conditions, but no difference between the numbers of trials to criterion in any other conditions. Reversals in this group increased after MK-801 by 38%, whereas SB-269970 alone had no effect, and its coadministration using either protocol restored performances to control levels (Figure 3).

Figure 3.

Reversal learning after injection of saline, SB-269970 alone (5HT7), MK-801, and a combination of the two drugs either coadministered (5HT7+MK) or injected at different time points (MK+5ht7; SB-269970 injected about 2 h after MK-801, right before the ASST). Group 2, N = 6.

Learning discrimination after an ID shift also deteriorated after MK-801 injection and the deficit was restored by SB-269970 (p < 0.05; Figure 1). For ED shift, the results showed a similar tendency, although the alterations did not reach the p = 0.05 level of significance. It should be noted, however, that the increase in average ED trials to criterion compared with ID in control ASST trials after saline injection was smaller than that in previous studies,^3,37 indicating that a single ID discrimination may have been insufficient to form a strong attentional set in our sample (Figure 1). Repeated IDs, i.e., multiple presentation of the same dimension in such cases, was shown earlier to strengthen the formation of attentional set in mice,^41–43 but to focus on reversal learning in this study we based our analysis of the original ASST paradigm developed for rats.³⁷ A previous study by Nikiforuk et al.,³ which specifically focused on ID/ED performance, showed a strong, significant 5-HT7 effect on ketamine-induced impairment in this task when a strong attentional set formed in the control conditions. This was verified by a high number of ED trials relative to ID.

The present findings are in agreement with previous studies which demonstrated SB-269970 efficacy in ameliorating schizophrenia-relevant reversal learning deficits in the subchronic phencyclidine model.^9,10 In this study, we showed for the first time that SB-269970 has a similar effect in the acute model of NMDA receptor blockade. SB-269970 reversed impairments of reversal learning induced by MK-801,^28–32 a noncompetitive NMDA receptor antagonist. Importantly, the effect on reversal learning was demonstrated using the traditional ASST paradigm,³⁷ i.e., the same task in which SB-2669970 reversed ketamine-induced impairment of attentional set-shifting,³ a different form of cognitive flexibility. Impaired reversal learning has been reported in different psychiatric conditions, such as depression²² and anorexia nervosa,^44,45 that are treated with compounds acting on the serotonergic system. The results indicate that 5-HT7 receptor mechanisms may have a specific contribution to the complex cognitive deficits found in a wide range of psychiatric diseases, which express different forms of cognitive inflexibility, even if direct evidence for antipsychotic-like or anxiolytic-like efficacy remains debated (cf., e.g., refs 11, 12 vs refs 16, 46).

The exact mechanisms require further investigations, which may include selective and nonselective 5-HT7 compounds and genetic studies. The interactions between NMDA and 5-HT7 receptor mechanisms are complex, organized on different levels of neural coordination. 5-HT7 antagonists may act selectively, normalizing glutamatergic transmission, and may also act on the level of neural networks. For example, SB269970 reversed working memory deficits induced by MK-801 (but not scopolamine), while it normalized MK-801-induced glutamate (but not dopamine) levels.⁴⁷ On the other hand, excitatory 5-HT (2c, 3, 6, 7) receptors strongly modulate cortical network activity, including rhythmic, synchronized neuronal firing,^48–53 involved in cognitive function. NMDA receptor blockade also drastically alters neural oscillations,^39,40 impaired also in human schizophrenia.^54,55 For compounds acting at the 5-HT7 receptor, such an effect was shown in preclinical studies^56,57 and in a recent human trial, in which neural communication in theta and gamma EEG bands was affected by treatment with vortioxetine.⁵⁸ This compound alleviates cognitive symptoms in animal models and in patients with depression.^21–24 Yet, on another level, a key role of 5-HT7 mechanisms was recently identified in maturation of the PFC–dorsal raphe circuit, implicating 5-HT7 receptors in the detrimental emotional effects associated with SSRI exposure during early postnatal life.⁵⁹

METHODS

Animals.

Fourteen male Sprague–Dawley rats were tested in the ASST. After a week of acclimatization, the rats were housed individually in separate cages. Single rat housing was required by the procedure of mild food restriction (10 g of food daily) applied for several days before the ASST. Training and testing occurred in the light phase of a 12 h light/dark cycle (lights on at 7 AM). All procedures were performed according to the NIH Guide and were approved by the BIDMC Institutional Animal Care and Use Committee.

Attentional Set-Shifting Task.

All ASST procedures closely followed those established by Birrell and Brown.³⁷

Apparatus.

The testing arena comprised of a dark plastic box (66 × 41 × 46 cm³) in which two bowls were placed in adjacent corners. In experiments of Group 2, an additional dark wall was used to separate the two bowls. Half of the arena, holding the bowls, was covered. The lid and the dark walls were used to prevent the rats from getting distracted by the surrounding environment during testing and to reduce potential anxiety. A camera was placed above the bowls for video recording of the animals’ behavior. A plastic box (25 × 20 × 30 cm³), open to the bottom, was used to contain the rat and to separate it from the bowls in between the trials. The bowls were scented and filled with sawdust on the bottom and digging medium on top, according to Table 1. To give each bowl a specific scent, odor essence was applied to a cotton pad and put in a plastic bag together with the bowl for 24 h before the first test. The bowls were renewed with both scent and digging medium after about four ASSTs. For the pretest, conducted the day before the ASST to make sure the rat was ready for testing, almond scented black gravel versus lemon scented black gravel was used for simple discrimination and lemon scented clear glass beads versus lemon scented black gravel was used for compound discrimination. The bait was buried in the middle of the sawdust layer beneath the digging medium, only in the correct bowl. On top of both bowls, food was sprinkled in order to reduce the capability of the rat to choose the correct bowl by smelling the bait.

Table 1.

Example of the Order of Discriminations Used in the ASST of Rats in Group 1^a

phase	dimensions		exemplar combinations
task	relevant	irrelevant	S+	S−
SD	Odor		clove	nutmeg
CD	odor	medium	clove + multicolor beads	nutmeg + multicolor beads
			clove + clear beads	nutmeg + clear beads
Revl	odor	medium	nutmeg + multicolor beads	clove + multicolor beads
			nutmeg + clear beads	clove + clear beads
ID	odor	medium	jasmine + brown paper	vanilla + brown paper
			jasmine + white paper	vanilla + white paper
Rev2	odor	medium	vanilla + brown paper	jasmine + brown paper
			vanilla + white paper	jasmine + white paper
ED	medium	odor	cinnamon + light foam	cinnamon + dark foam
			patchouli + light foam	patchouli + dark foam
Rev3	medium	odor	cinnamon + dark foam	cinnamon + light foam
			patchouli + dark foam	patchouli + light foam

^aThe correct pot is signified by S+ and the incorrect by S−. For each discrimination problem, the correct choice was paired with either exemplar from the irrelevant dimension. In the ID and ED phases, novel exemplars for each dimension was introduced. In Group 2, the odor and medium were switched as relevant and irrelevant dimensions (see Design).

Habituation and Training.

After habituation to the testing arena and the bowls containing the food, the rats were trained on a series of simple discriminations (SD). The aim of the training was for the rats to learn to associate a food reward with an odor or/and a digging medium. The same pair of stimuli was used for all rats during the training, and the positive and negative cues were presented randomly and equally for each rat. The stimuli used in the training were not used later in the actual test. Rats are able to learn to dig reliably in approximately 4–7 days according to the following schedule:

Days 1–2:

The arena was covered halfway by the lid with mounted camera. The rat was placed and left alone to explore the arena for about 5 min without a bowl in it. After about 5 min a bowl filled with bedding (sawdust) was placed into one corner of the arena, on the side covered with the lid. Roughly five pieces of food was placed in the center of the bowl, on top of the bedding. Only one bowl was used at this point. The goal was only to get the rat to eat the food; it was not timed. The corner, in which the bowl was placed, was randomized so that the rat would not form a side preference. Training stopped after 30 min or after the rat ate 10 pieces of food. These steps were repeated until the rat ate 10 pieces of food during one 30 min session.

Days 2–3:

After the rat completed the first phase of eating, 10 uncovered pieces of food within one 30 min session, the process of burying the food was initiated little by little. The rat was left in the arena to adjust for about 5 min. One piece of food was placed in the center of the bowl on top of the bedding with the corner randomized. Still only one bowl was used at this stage. When the rat managed to reach about four uncovered pieces, the food was gradually concealed more and more over the next trials until it was deeply buried. The goal was to get the rat to eat five uncovered pieces and five buried pieces.

Days 3–4:

The rat was left to adjust to the arena for about 5 min before training commenced. At this point, every piece of food was buried, and the corner was randomized according to the learning to dig (LTD) schedule sheet. The goal was to get the rat to eat 10 buried pieces, but still without timing.

Days 4–7:

The rat was placed in the arena for about 5 min, and after that the holding box was introduced. The rat was placed under the holding box for about 2 min while the baited bowl was placed in the randomized corner according to the LTD schedule sheet. The holding box was relatively large, almost the size of the rats’ home cage (16″ × 20″, 8.25″ H) so the animals could get used to it within the next 1 to 2 days. At this stag,e the rat had only 90 s to find and eat the food during the first four trials and only 60 s during trials five and above. This training phase was continued until the rat retrieved 10 buried pieces within the time limit in a row (consecutive). Once the rat reached the goal of finding 10 buried food pieces within the time limit in consecutive trials, it was considered ready for the exemplar phase, or the ASST pretest. The exemplar, as stated above, consisted of almond versus lemon (odor discrimination) and glass beads versus black gravel (digging medium discrimination). The exemplar phase has to be conducted the day before each ASST to make sure the rat is ready for testing. The complete ASST should not be conducted unless the rat completed the exemplar stage successfully. Exemplars follow the same rules as the ASST; six consecutive correct trials are required to pass.

ASST Testing.

The procedure was adopted by Birrell and Brown.³⁷ Testing consisted of seven phases; details are shown in Table 1, and a schematic overview can be seen in Figure 4. The first four trials of each phase were exploratory; the rat had 90 s to complete the trial, and it could dig in the wrong bowl without ending the trial. If the rat dug in the wrong bowl first but later dug in the correct one, the trial would still be marked as a failure. During trial five and onward, the rat only had 60 s to complete and the trial would end immediately if it dug in the wrong bowl. If time ran out without the rat digging in any of the bowls, the trial would be marked as an omission. Omissions did not restart a streak of consecutive correct trials. Any noticeable displacement of digging medium with the nose or paw would be considered as digging. The rat was allowed to sniff and touch the incorrect bowls as long as no evident displacement of digging medium occurred. When six consecutive correct trials were reached, that phase of the test was complete and the rat could move on to the next one. Between each set of novel exemplars, i.e., between Rev1 and ID as well as between Rev2 and ED, the rat got a break for 5–10 min in order to maintain its interest and to let it drink water. During the break, the rat was moved back to its home cage while the new bowls were prepared.

Figure 4.

Schematic overview of the ASST procedure. Progression of discriminations from simple (SD) and compound discriminations (CD) through intra- (ID) and extradimensional (ED) shifts alternated with reversals (Rev1, −2, and −3) after different discriminations, CD, ID, and ED. The relevant dimension was odor in the first phases switched to digging material in ED and the following Rev3. The rewarded and unrewarded stimuli in the relevant dimensions were switched in reversals, compared with the immediately preceding stage.

Compounds.

MK-801 (NMDA receptor antagonist) was purchased from Tocris through R&D Systems, Inc. (Minneapolis, MN, USA) and injected in a dose of 0.2 mg/kg of body weight, which was shown in prior studies to elicit schizophrenia-like symptoms in rats.^39,40 SB-269970 was purchased from AbCam (Cambridge, MA, USA); the compound is a 5-HT7 receptor antagonist with a pK_i of 8.3 and exhibits >50-fold selectivity against other receptors. It was injected in a dose of 2 mg/kg, i.e., in the range used in prior behavioral studies (1–3 mg/kg^3,9,10). In vivo detection of 5-HT7 antagonism in rats showed that 1–30 mg/kg SB-269970 dosedependently, and almost completely, inhibited serotonin-induced hypothermia with an ED₅₀ of 2.96 ± 0.2 mg/kg.^16,60 This reaction was also shown to be missing in 5-HT7 receptor knockout mice.⁶¹

Design.

The rats were divided into two groups and tested according to one of two experimental designs. In the first group (n = 8), each rat was tested on the ASST three times under different conditions. A week after a control ASST to establish baseline with no injection, each rat was injected once with MK-801 and once with MK-801 and SB-269970 before retesting in the ASST. The order of drug injections was randomized: in four rats, MK-801 was tested before the MK+SB-269970 combination; in the four others, the order was reversed. At least 1 week was allowed between the injections. ASST after MK-801 injection started as soon as the rat could pick up, hold, and eat a piece of food, approximately 2–3 h after injection. ASST after the MK-801 and SB-269970 coadministration started with the same delay, i.e., approximately 2–3 h.

Rats in the second group (n = 6) were tested six times each, after injection of saline, MK-801, SB-269970 alone, after coadministration of MK801 and SB-269970, and after coadministration followed by a second injection of SB-269970 about 2 h later, i.e., immediately before starting the ASST. Saline injection was tested once at the beginning of the experiment and repeated at the end of the series in each rat whereas the order of the other injections was randomized. Using odor and digging medium was also randomized so either odor or medium was used as the relevant dimension during the first five stages of ASST in Group 1 and 2, respectively (Table 1).

Statistics.

ASST data were analyzed using two-way ANOVAs to assess drug effects, followed by pairwise posthoc Bonferroni comparisons using OriginPRO software. Alpha was set as p < 0.05.

ABBREVIATIONS

ASST

attentional set-shifting task

SD

simple discrimination

CD

compound discrimination

ID

intradimensional

ED

extradimensional

Rev

reversal

PFC

prefrontal cortex

5-HT7

5-hydroxytryptamine 7

NMDA

N-methyl- d-aspartate

D2

dopamine 2