Acute psilocybin enhances cognitive flexibility in rats

Abstract

Psilocybin has been shown to improve symptoms of depression and anxiety when combined with psychotherapy or other clinician-guided interventions. To understand the neural basis for this pattern of clinical efficacy, experimental and conceptual approaches that are different than traditional laboratory models of anxiety and depression are needed. A potential novel mechanism is that acute psilocybin improves cognitive flexibility, which then enhances the impact of clinician-assisted interventions. Consistent with this idea, we find that acute psilocybin robustly improves cognitive flexibility in male and female rats using a task where animals switched between previously learned strategies in response to uncued changes in the environment. Psilocybin did not influence Pavlovian reversal learning, suggesting that its cognitive effects are selective to enhanced switching between previously learned behavioral strategies. The serotonin (5HT) 2 A receptor antagonist ketanserin blocked psilocybin’s effect on set-shifting, while a 5HT2C-selective antagonist did not. Ketanserin alone also improved set-shifting performance, suggesting a complex relationship between psilocybin’s pharmacology and its impact on flexibility. Further, the psychedelic drug 2,5-Dimethoxy-4-iodoamphetamine (DOI) impaired cognitive flexibility in the same task, suggesting that this effect of psilocybin does not generalize to all other serotonergic psychedelics. We conclude that the acute impact of psilocybin on cognitive flexibility provides a useful behavioral model to investigate its neuronal effects relevant to its positive clinical outcome.

Introduction

Recent clinical trials indicate that psilocybin is an effective treatment for symptoms of several psychiatric disorders, including anxiety and depression [1–5]. A unique aspect of these studies is that sustained therapeutic efficacy is observed when one or a few doses of psilocybin are administered during clinician-guided facilitation or intervention [6–11]. To date, clinical data showing that psilocybin has therapeutic efficacy as a stand-alone treatment is lacking, suggesting that its effects may be, at least in part, due to the combination of therapeutic intervention and exposure to acute psilocybin. Accordingly, multiple ongoing clinical trials are assessing the efficacy of joining psychotherapy or cognitive behavioral therapy with psilocybin treatment (e.g., NCT05227612). While the pharmacology of psilocybin is relatively well characterized, how this translates to its clinical effects remains unclear. What are the mechanisms that underlie psilocybin’s clinical efficacy when combined with psychotherapy or other clinician-guided interventions? Though several ideas have been proposed [12], we focused on the hypothesis that acutely psilocybin creates a cognitive state that improves therapeutic intervention by relaxing prior beliefs, promoting flexible thinking, and increasing cognitive flexibility [1, 13, 14].

Cognitive flexibility is the ability to switch between mental processes in order to appropriately adapt behavior to changes in the environment. Impairments in cognitive flexibility are a prominent feature of many psychiatric disorders, including major depression (MDD) and substance use disorders [15–19], which have shown a positive response to psilocybin-assisted treatment. Thus, the capacity to improve cognitive flexibility in order to facilitate breaking out of rigid behavioral patterns and beliefs is a plausible mechanism by which psilocybin may exert its therapeutic benefits [1, 14, 20].

Preclinical studies are critical for advancing our mechanistic understanding of the therapeutic efficacy of psilocybin. There is a growing appreciation that traditional behavioral tests used to study antidepressant effects do not reliably translate to effects in human populations [21]. While several traditional preclinical approaches to study depression-like behaviors in rodents have been used in conjunction with psilocybin [22, 23], little is known about the acute effect of this drug on cognitive flexibility and related constructs. Other work has examined the acute effects of psychedelics on cognitive flexibility, but none of these studies used psilocybin [24, 25]. We hypothesize that the acute effects of psilocybin may be inducing a state of increased cognitive flexibility that enhances the therapeutic response during clinician-assisted sessions.

To assess cognitive flexibility, we used a set-shifting task, which tests an animal’s ability to modify behavior, among competing and previously learned strategies, in response to changes in environmental contingencies. We find that acute psilocybin treatment improves set-shifting behavior without influencing appetitive or aversive learning. Psilocybin’s effect on cognitive flexibility was diminished by a 5HT2A but not by a 5HT2C serotonin receptor antagonist. Consistent with other studies showing impaired cognitive flexibility with psychedelics other than psilocybin [24, 25], we find that 2,5-Dimethoxy-4-iodoamphetamine (DOI), worsened set-shifting ability. These data support the idea that enhancing cognitive flexibility is a mechanism underlying the clinical effects of psilocybin.

Materials and methods

Animals

Experiments were performed in accordance with NIH’s Guide to the Care and Use of Laboratory Animals and were approved by the Oregon Health and Science University Institutional Animal Care and Use Committee. Long-Evans rats of both sexes were bred in house or purchased from Charles River Laboratories (CRL, Wilmington, MA) at 8 weeks of age. Rats (n = 49, 24 males and 25 females) were pair-housed in same-sex pairs in a room with controlled humidity and temperature under a 12 h reverse light/dark cycle with lights off at 9:00 am. All behavioral tasks were conducted during the active dark phase. After acclimation and handling, animals were food restricted to 85–90% of their free-feeding weight before training on behavioral tasks began (see Supplementary Material for details of training procedures).

Drugs

Psilocybin (received from Usona Institute and NIDA Investigational Drug and Material Supply Program) and DOI (Sigma-Aldrich, cat #D101) were dissolved in saline to a concentration of 1 mg/ml. Ketanserin (Sigma-Aldrich, cat #S006) and SB242084 (Tocris, cat #2901) were dissolved in 10% DMSO to a concentration of 1 mg/ml. Solutions were prepared on the day of injection when possible and otherwise stored at −20 °C for up to a month. All injections were intraperitoneal. The dosage of 1 mg/kg for psilocybin and DOI has been shown to reliably elicit the head-twitch response, a behavioral marker of psychedelic activity [23, 26]. The chosen dose of ketanserin abolishes the head-twitch responses produced by psilocybin [23]. We selected a relatively high dose of SB242084, which has behavioral effects in decision-making tasks [27, 28]. Saline, DOI or psilocybin were injected 20 min before behavioral testing. Antagonist were injected 10 min before the main treatment (30 min prior to testing). Animals received multiple injections (min 1, max 4) and were given at least 2 weeks of washout between injections. Treatments were not repeated in the same animals and given in random order (Supplementary Table S2).

Set-shifting task

Animals were tested on the extradimensional set-shifting task in daily sessions as previously described [29–31] (Fig. 1a; see Supplementary Materials for training procedure and testing details). Briefly, on each trial one of two nose-poke ports became illuminated. Rats were required to respond to one of the ports according to a “Light” rule, where the lit port was rewarded, or a “Side” rule, where the port on a specific side was rewarded, regardless of which one was lit. If and when rats reached a criterion of 10 consecutive correct responses, the rule changed. Each session lasted until criterion was reached on four rule sets, or until 45 min elapsed. Three separate cohorts of animals were used in this study, and behavior was consistent across cohorts (Supplementary Fig. S1).

Fig. 1. Set-shifting task.

a Schematic representation of the operant set-shifting task. b Example sessions for one animal. Top, two sessions in the early stages of training. Bottom, two sessions during stable behavior. Vertical ticks denote the animal’s choice on each trial (Left or Right; black ticks, correct responses; red ticks, incorrect responses). Note that which port is illuminated on each trial is not shown. The current rule for each trial is shown in the colored bar at the top (yellow, Light rule; pink, Side rule). SL = side (left); SR = side (right); L1 = first light set; L2 = second light set. c Number of sessions until performance stabilized for every animal used in the set-shifting task (n = 35, 18 males and 17 females). Black open circles indicate the point at which each animal reached stability (see Methods and Supplementary Materials for details). Gray solid circles indicate the first session in which that animal received a treatment injection (saline or drug). d Schematic of drug administration and task timeline. Three sessions of stable behavior were used as a baseline for each animal. For main treatment sessions (top) injections were given 20 min prior to the start of the session. For antagonist sessions (bottom), the antagonist drug was injected 10 min prior to the main treatment.

Reversal learning task

The reversal learning task was a modified version of a previously characterized task [32] (see Supplementary Materials for more details). Briefly, two different conditioned stimuli were associated with an appetitive or aversive unconditioned stimulus (US). Animals performed ten task sessions during which 50 stimulus-outcome associations for each kind of CS-US pairing were presented. On the sixth session, CS-US contingencies were reversed such that the CS that previously signaled an appetitive outcome now predicted the aversive one, and vice-versa. Injections of psilocybin (1 mg/kg) or an equivalent volume of saline were given 20 min before starting the task on the reversal session (session 6). We chose to give only one dose of psilocybin for consistency with the set-shifting experiment.

Analysis and statistical tests

To assess set-shifting performance we calculated trials to criterion, number of completed sets, and fraction of correct responses, for 3 days of baseline and for the drug treatment session (Fig. 1d). The average trials to criterion value was obtained by calculating the number of trials needed to reach 10 consecutive correct responses on each set and averaging across all completed sets in a session. Note that this excludes sets that were not completed and therefore may be artificially low when animals quickly complete one set but then fail to shift their behavior to the new rule. This would result in a low value for the average trials to criterion, but it cannot be taken as evidence of effective set-shifting behavior. For this reason, we used a combination of trials to criterion, number of completed sets, and fraction of correct responses to assess set-shifting performance. The fraction of correct responses was calculated by combining trials from all sets. We also calculated “streaks”, defined simply as any bout of consecutive correct responding. Lastly, we computed response time, defined as the time elapsed between the stimulus onset (lit nose-poke port) and action choice (nose-poke action). For the reversal learning task, we evaluated the behavior by computing the R-value [32]. This is defined for each stimulus in a session as the total number of responses in the food port during the cue presentations, divided by total number of responses in the food port during the second half of the ITIs. Because for each CS there are twice as many ITIs as there are CS presentations, a R-value of 0.5 corresponds to no conditioning. Larger values indicate appetitive conditioning, smaller values indicate aversive conditioning.

Statistical analyses were performed using GraphPad Prism software (GraphPad, San Diego, CA) or custom Python scripts. For all statistical tests we used t-tests when data were normally distributed and rank-sum or sign-rank tests when they were not. Two-way ANOVA (or mixed-effects analysis, when some data were missing) followed by Sidak or Dunnett’s post-hoc tests was used to compare the effect of two factors on behavior. For the reversal learning task, we used a mixed-effects model followed by Sidak post-hoc tests. The significance threshold for all tests was set to 0.05.

Results

Characterization of behavior in extradimensional set-shifting task

The effect of psilocybin on cognitive flexibility was tested using a previously characterized operant set-shifting task [29, 31]. Adult rats of both sexes (n = 35, 18 males and 17 females) were trained in the task and all animals learned the task well (Fig. 1; see Methods and Supplemental Methods for details). After animals reached stable behavior (Fig. 1c; see Supplementary Methods), they received their first i.p. injection 20 min before behavioral testing (Fig. 1d; see Supplementary Methods).

Acute psilocybin improves performance in set-shifting task

Psilocybin (1 mg/kg, i.p.) improved performance on the set-shifting task, as indicated by a significant decrease in the average number of trials rats needed to reach criterion (Fig. 2a, p = 0.005, n = 15; see also Supplementary Fig. S2a, b). A comparable number of sets was completed with psilocybin treatment and baseline (Fig. 2b, p = 0.546), despite significantly slower response times (Fig. 2c, p = 0.008). Psilocybin did not change the fraction of correct responses (Fig. 2d, p = 0.188) but correct responses can be made without reaching the criterion of 10 consecutive ones. We, therefore, analyzed the effect of psilocybin on a “streak”, defined as any series of consecutive correct responses. We observed that animals receiving psilocybin performed longer without making an incorrect response, resulting in fewer and longer streaks on average (Fig. 2e, f; number of streaks, p = 0.025; average streak length, p = 0.041). These analyses suggest that psilocybin selectively reduces incorrect responses occurring after a series of correct ones, thus increasing the likelihood of reaching criterion on the task.

Fig. 2. Psilocybin improves performance in the set-shifting task.

a Average trials-to-criterion across all sets in the baseline period (PRE, open gray circles) and in the acute psilocybin condition (PSI, blue circles, n = 15). ***p = 0.0005, paired t-test. b Number of completed sets in baseline and psilocybin conditions, p = 0.546, Wilcoxon sign-rank test. c Response time in baseline and psilocybin conditions. **p = 0.008, Wilcoxon sign-rank test. d Fraction of correct responses across all sets in baseline and psilocybin conditions. e Number of streaks of consecutive correct responses averaged across all sets, in baseline and psilocybin conditions. *p = 0.025, Wilcoxon sign-rank test. f Average length of streaks averaged across all sets, in baseline and psilocybin conditions. *p = 0.034, paired t-test. g Average trials-to-criterion for baseline (PRE, gray open circles) and saline conditions (SAL, brown circles, n = 14). h Number of completed sets in baseline and saline conditions. i Fraction of correct responses across all sets in baseline and saline conditions.

We performed a number of additional analyses to investigate the effects of psilocybin in detail. First, we note that the effect of psilocybin on trials to criterion was most pronounced in the first two sets (Supplementary Fig. S2a), possibly due to it wearing off in sets 3 and 4. To confirm that animals completed the first two sets while on the acute effects of psilocybin, we computed the time to criterion for each set during baseline and psilocybin days. Time to criterion values were similar in baseline and psilocybin days. Furthermore, nearly all animals completed the first two sets within 30 min of starting the task, i.e., within 50 min of the injection (Supplementary Fig. S2e), which is well within the time window for psilocybin’s acute effects. Secondly, we analyzed Light- and Side-rule sets separately to ask whether psilocybin differentially affected performance depending on rule, rather than order of sets. These comparisons did not reach significance when looking at all four sets (data not shown; Light, p = 0.059; Side, p = 0.051), but they were both significant when we restricted the analysis to the first two sets (Supplementary Fig. S2f; Light, p = 0.026; Side, p = 0.024). We conclude that psilocybin had a similar effect on both Light and Side sets. We also analyzed perseverative and regressive errors for the psilocybin animals. Here we define perseverative errors as the number of trials after a rule switch where the animal chooses the wrong rule before a streak of X consecutive correct responses in the new (current) rule (where X is a threshold that can vary). We similarly define regressive errors as the number of previous rule choices made after X consecutive correct responses in the current rule. We found no evidence of differences between baseline and psilocybin conditions (Supplementary Table S1). Finally, we note that data for both sexes were combined as no sex differences were found (Supplementary Fig. S2g).

In the analyses above, we compared behavior on the day of treatment to the average performance in the previous three days (baseline days). It is critical for this analysis that the behavior in the baseline period is stable. To ensure that our stability criteria (see Supplementary Methods) resulted in stable baseline performance, we plotted trials to criterion, completed sets and fraction of correct responses on baseline, and on treatment days, for all psilocybin-treated animals. The behavior of individual animals oscillated within a consistent range across the baseline sessions, and the average across the cohort was stable (Supplementary Fig. S2b). We also aimed to control for rule preference. For each animal we split sessions by whether their starting rule (i.e., in the first set) was a light or side rule, and quantified behavioral metrics for stable sessions performed by that animal. This analysis showed that for the psilocybin-treated cohort, performance as a group was not biased by the starting rule in each session (Supplementary Fig. S2b).

Animals receiving vehicle saline injections completed a similar number of sets as they did during the baseline period (Fig. 2h, p = 0.237, n = 14; see also Supplementary Fig. S2c, d), but they did not show any difference in the trials-to-criterion metric on the day of injection (Fig. 2g, p = 0.703). They also showed no change in the fraction of correct responses (Fig. 2i, p = 0.463). We used the saline-treated group as a control and compared trials to criterion during baseline for all treatment groups to the baseline value for the saline group. We found no difference, indicating that all treatment groups had similar pre-treatment performance in the task (Supplementary Fig. S2h; two-way ANOVA followed by Dunnett’s test, all p > 0.35). We also compared each group’s treatment data with that of the saline group and found a significant difference for psilocybin (Supplementary Fig. S2h; two-way ANOVA followed by Dunnett’s test, p = 0.005), among others (see below).

Psilocybin does not impact performance on a reversal learning task

Next we asked whether psilocybin can facilitate learning of changes in environmental contingencies, a construct generally assessed in reversal learning tasks. Reversal learning and set-shifting involve different neural mechanisms, consistent with the substantial difference in the cognitive demands of each task [33, 34]: whereas reversal learning involves updating cue-outcome (or action-outcome) associations, set-shifting requires no new learning but rather tests the capacity to adaptively switch between known behavioral strategies. To test reversal learning, we used another previously characterized task [32]. This task has the advantage of assessing reversal learning related to both appetitive and aversive outcomes so that the impact of psilocybin on affective context could also be evaluated. Drug-naive and untrained rats were exposed to two different cue-stimulus pairings over 10 task sessions. The cue was either a tone or a flashing light, and each was paired either with a mild foot-shock or a food reward (Fig. 3a; see Supplementary Methods for details). The pairings were switched on session 6. Rats were injected with psilocybin (1 mg/kg, n = 7, 3 males and 4 females) or an equivalent volume of saline (n = 6, 3 of each sex) 20 min before the reversal session (Fig. 3b).

Fig. 3. No effect of psilocybin in the reversal learning task.

a Schematic diagram of the Pavlovian reversal learning task. b Structure of the reversal learning task, including time of injection. c R-value comparison of responding to the initially appetitive cue (App > Av) between saline-treated (open circles, n = 6) and psilocybin-treated (solid circles, n = 7) animals. Red arrow indicates the session in which contingencies change and treatment is administered. Mixed-effects analysis: main effect of Treatment, p = 0.472; interaction effect, Session x Treatment, p = 0.179. d As in (c), for the initially aversive cue. Mixed-effects analysis: main effect of Treatment, p = 0.834; interaction effect, Session x Treatment, p = 0.993. e R-value in early (first 10) and late (last 10) trials in session 6 of the reversal task, when the reversal happened, for the initially appetitive cue. All individual animals are shown. Two-way ANOVA: main effect of Timepoint, p = 0.009; interaction effect, Treatment x Timepoint, p = 0.487. f As in (e), for the initially aversive cue. Two-way ANOVA: main effect of Timepoint, p = 0.371; interaction effect, Treatment x Timepoint, p = 0.754.

Both saline and psilocybin groups learned the cue-outcome contingencies, and after the reversal the previously aversive cue became appetitive for both groups, and vice versa for the initially appetitive cue (Fig. 3c, d). No differences were found when comparing responses to each contingency pairing for the two treatment groups (Fig. 3c, d). To investigate differences in within-session learning during the reversal session, we plotted the R-value in early (first 10) and late (last 10) trials in that session, for each animal, cue type and treatment group (Fig. 3e, f). Both saline- and psilocybin-treated animals showed evidence of learning for the previously appetitive cue (Fig. 3e; two-way ANOVA, main effect of Timepoint, p = 0.009), but there were no differences between groups (interaction effect Treatment x Timepoint, p = 0.487). We also found no differences in within-session learning between groups for the previously aversive cue (Fig. 3f; two-way ANOVA, interaction effect Treatment x Timepoint, p = 0.754).

These data show that psilocybin does not influence this form of reversal learning, regardless of the nature of the outcome, indicating specificity in its cognitive effects. Our results suggest that while acute psilocybin improves switching between known behavioral strategies, it does not improve learning of changes in environmental contingencies.

Enhanced cognitive flexibility by psilocybin does not generalize to the psychedelic DOI and is blocked by ketanserin

To test whether enhanced flexibility could result from treatment with a different psychedelic agent, we treated animals with DOI, a commonly used psychedelic compound which is an agonist at 5HT2A receptors. DOI (1 mg/kg, i.p.) did not improve set-shifting performance (Fig. 4a, p = 0.112, n = 12; see also Supplementary Fig. S3). Moreover, rats completed significantly fewer sets when treated with DOI compared to the baseline period (Fig. 4b, p = 0.012), with most completing only one or no sets at all. The fraction of correct responses was also lower compared to baseline (Fig. 4c, p = 0.005). In the comparison to saline, the DOI group showed a significant effect (Fig. S2h, p = 0.013). However, we argue that the low value for trials to criterion in the DOI group is largely due to artificially low numbers for that metric when animals complete very few sets, as explained above. Most animals treated with DOI only completed one set or none at all. Overall this represents a clear deficit in performance of extradimensional shifts. Thus, the effect of psilocybin on set-shifting does not generalize to all other serotonergic psychedelics, suggesting that psilocybin may have a unique profile of cognitive effects.

Fig. 4. Treatment with DOI impairs performance in the set-shifting task.

a Average trials-to-criterion for baseline (PRE, gray open circles) and DOI conditions (red circles, n = 12). b Number of completed sets in baseline and DOI conditions. *p = 0.012, Wilcoxon sign-rank test. c Fraction of correct responses averaged across all sets in baseline and DOI conditions. **p = 0.005, Wilcoxon sign-rank test.

Psilocin, the active metabolite of psilocybin, is a partial agonist of both 5HT2A and 5HT2C receptors. Activation of the former is thought to be responsible for the subjective effects of psilocybin and other psychedelics, so we investigated whether agonism of 5HT2A receptors played a role in the cognitive effects we observed. We pre-treated rats with the 5HT2A antagonist ketanserin (1 mg/kg, i.p.) 10 min before injection of either psilocybin or saline. Ketanserin blocked the effect of psilocybin on cognitive flexibility (Fig. 5a−c, n = 14; trials to criterion, p = 0.231; completed sets, p = 0.688; fraction correct, p = 0.469; see also Supplementary Fig. S4). This was also evident when comparing the ketanserin + psilocybin group to the saline control group (Fig. S2h, p = 0.431). We therefore conclude that the effects of psilocybin are at least partly mediated by its action at 5HT2A receptors.

Fig. 5. Pre-treatment with the 5HT2A antagonist ketanserin blocks psilocybin’s effect on cognitive flexibility.

a Average trials-to-criterion for baseline (PRE, gray open circles) and ketanserin-psilocybin condition (KET + PSI, orange circles, n = 14). b Number of completed sets in baseline and ketanserin-psilocybin conditions. c Fraction of correct responses averaged across all sets in baseline and ketanserin-psilocybin conditions. d Average trials-to-criterion for baseline (PRE, gray open circles) and ketanserin-saline conditions (KET + SAL, pink circles, n = 11). *p = 0.026, paired t-test. e Number of completed sets in baseline and ketanserin-saline conditions. f Fraction of correct responses averaged across all sets in baseline and ketanserin-saline conditions. *p = 0.05, paired t-test. g Average trials-to-criterion for baseline (PRE, gray open circles) and SB242084-psilocybin conditions (SB + PSI, green circles, n = 11). *p = 0.032, Wilcoxon sign-rank test. h Number of completed sets in baseline and SB242084-psilocybin conditions. i Fraction of correct responses averaged across all sets in baseline and SB242084-psilocybin conditions. j Average trials-to-criterion for baseline (PRE, gray open circles) and SB242084-saline conditions (SB + SAL, teal circles, n = 11). k Number of completed sets in baseline and SB242084-saline conditions. l Fraction of correct responses averaged across all sets in baseline and SB242084- saline conditions. *p = 0.031, paired t-test.

Rats injected with ketanserin followed by saline also showed a performance improvement comparable to psilocybin-treated animals. They required fewer trials to reach criterion (Fig. 5d, p = 0.026, n = 11), while completing a similar number of sets as during baseline (Fig. 5e, p = 0.680), and showed a subtle but significant increase in the percent of correct trials (Fig. 5f, p = 0.050). We also found a significant effect when comparing to the saline control group (Fig. S2h, p = 0.0004). This effect of ketanserin is consistent with a previous report [35] and suggests a complex relationship between 5HT2A receptor function and cognitive flexibility.

Psilocin also binds to 5HT2C receptors. While ketanserin binds more strongly to 5HT2A receptors, it has moderate affinity at 5HT2C receptors. We, however, found that the 5HT2C antagonist SB242084 (1 mg/kg, i.p.) did not impact psilocybin’s enhancement of cognitive flexibility in the set-shifting task (Fig. 5g–I, n = 11; trials to criterion, p = 0.0322; completed sets, p = 0.484; fraction of correct responses, p = 0.249; see also Supplementary Fig. S5). Injection of SB242084 followed by saline produced a trend towards impaired performance (Fig. 5j–l, n = 11; trials to criterion, p = 0.062; completed sets, p = 0.762; fraction of correct responses, p = 0.031). The same results were seen when comparing the SB242084 groups to the saline animals (Fig. S2h; SB242084 + saline, p = 0.999; SB242084 + psilocybin, p = 0.010). Thus, the improvement in cognitive flexibility seen with acute psilocybin treatment is likely independent of its action at 5HT2C receptors.

Lastly, we analyzed the impact of each treatment on the average number and average length of streaks. We find that the results match the trails to criterion analysis: treatments that reduced trials to criterion also decreased the number of streaks, while increasing their average length (Supplementary Fig. S6).

Discussion

Psilocybin has received considerable attention for its therapeutic potential in treating anxiety and depression when combined with psychotherapy or other clinician-assisted interventions [1, 2, 4–8]. The biological mechanism underlying this effect is not well understood, but one interesting hypothesis is that psilocybin increases cognitive flexibility [1]. We investigated this idea and the pharmacological basis for psilocybin’s action using rats trained on an extradimensional set-shifting task. We find that psilocybin improves the ability to flexibly switch between learned action patterns. In contrast, psilocybin does not affect learning of new environmental contingencies. This effect of psilocybin did not generalize to the serotonergic hallucinogen DOI, which impaired cognitive flexibility, highlighting differences between psilocybin and other psychedelics. We further found that the 5HT2A/C antagonist ketanserin, but not the 5HT2C-selective antagonist SB242084, blocked the psilocybin-induced improvement in flexibility. These findings demonstrate a possible mechanism for the therapeutic action of psilocybin and provide a behavioral model for future investigation of the biological basis of the clinical efficacy of acute psilocybin or related novel compounds.

Psilocybin selectively improves switching between known behavioral strategies

Cognitive flexibility in rodents has been studied using two experimental approaches: reversal learning tasks and set-shifting tasks. These two approaches, however, assess different constructs. Reversal learning requires animals to learn new stimulus-outcome or response-outcome associations, in Pavlovian and instrumental versions of the task respectively [34]. Set-shifting tasks, similar to the test we used here, instead involve an attentional and behavioral shift based on the representation of previously learned strategies [33]. While action-outcome associations do change in our task, the primary driver of behavior is the choice between internalized representations of task rules and not newly learned associations. We note that it is possible that psilocybin would have different effects in an instrumental version of the reversal learning task, or one that does not include stimuli of both positive and negative valences, which may be closer to updating tasks used in humans. Nevertheless, our results support the idea that psilocybin’s acute effects differentially affect flexibly choosing between known strategies versus updating of environmental contingencies.

Our findings are consistent with human studies with ayahuasca [36] and psilocybin [20], and have interesting implications for understanding psilocybin’s effect on brain function. Set-shifting and reversal learning involve distinct neural circuits [37, 38]. A large number of studies in humans, non-human primates and rodents have identified the orbitofrontal cortex (OFC) as the primary region required for reversal learning [34, 39–41], while medial prefrontal cortex (mPFC), and particularly prelimbic cortex (PrL), is necessary for extradimensional set-shifting [42–46]. Neuronal dynamics in the PrL represent task rules, and these representations shift when animals adapt their behavior [30, 31, 47–49]. The anterior cingulate cortex (ACC), another subdivision of mPFC, has also been implicated in flexible behavior, though its specific role in the rat is likely different than that of the PrL [38, 50–52]. In humans, functional connectivity of the ACC is affected by psilocybin, providing a potential link to changes in flexibility, though the mechanism remains unclear [20].

At the level of neuronal networks, psilocybin may impact set-shifting by modulating neuronal activity in mPFC both directly, via activation of 5HT receptors, as well as indirectly via its effects on neuromodulatory regions that project to frontal cortex. Recent results that show a preeminent role of inhibitory interneurons in mediating strategy switching [53, 54] and increased expression of activity-dependent genes in inhibitory neurons in the mPFC following psilocybin administration [55] support this notion. Psilocybin also impacts the activity in the dorsal raphe nucleus (DRN) and the locus coeruleus (LC) [56–60]. These two inter-connected regions reciprocally regulate their firing, and provide serotonergic and noradrenergic inputs to cortical regions. These two neurotransmitters systems have been strongly implicated in flexible decision-making and set-shifting [61–64]. Studies with psilocin and related compounds such as lysergic acid diethylamide (LSD) suggest that psilocybin decreases DRN firing [56, 58, 65, 66], which may play a role in modulating behavioral flexibility. Finally, systemic administration of psilocybin likely influences activity in other subcortical regions that are critical for motivated behavior, such as the ventral tegmental area (VTA) [67]. Future studies are required to understand how psilocybin and other psychedelics affect network dynamics within the DRN, LC and VTA, and in the broader context of their projections to OFC and mPFC circuits.

Pharmacological site of action of psilocybin

We found that psilocybin-mediated increases in cognitive flexibility in rats were prevented by blocking 5HT2A receptors, but not 5HT2C receptors. While there is a substantial literature on the impact of 5HT receptors on cognitive flexibility tasks, results are often contradictory [61]. Consistent with our experiments with 5HT2A and 5HT2C antagonists, ketanserin has been shown to improve strategy switching in a spatial extra-dimensional shifting task, while SB242084 did not affect performance in the same task [35]. In reversal learning tasks, results are mixed with 5HT2A antagonists or agonists impairing [24, 27, 68] or improving [69] performance, and 5HT2C antagonists having no effect or enhancing [70] performance.

Psilocybin and ketanserin both improved set-shifting performance, but this improvement was absent with concurrent administration of both drugs. There are several possible explanations for this effect. First, activating and blocking 5HT2A receptors may be leading to different downstream circuit changes that each result in improved flexibility. These would have to be competing circuits, given that concurrent administration of both drugs has no effect. Alternatively, the two opposing manipulations may be converging on the same output circuit. It is worth noting that psilocybin has moderate affinity for 5HT1A receptors, the activation of which is known to depress firing in the DRN, thus likely causing lower extracellular 5HT levels in downstream areas. Studies in 5HT-depleted rodent models however generally show either no effect [71, 72] or impairments in flexible behavior [73–75], in contrast with our data. It is possible that psilocybin’s action at 5HT2A receptors in this context can compensate for the decrease in endogenous 5HT and increase behavioral flexibility. Conversely, ketanserin’s antagonistic effect at 5HT2A receptors may result in increased 5HT levels by disrupting inhibitory feedback control of DRN neurons [76]. Therefore, ketanserin may be beneficially impacting flexibility by increasing endogenous 5HT levels, akin to commonly used antidepressants, such as selective serotonin reuptake inhibitors (SSRIs), an effect which has also been shown to improve performance in reversal learning tasks [69, 77]. Finally, ketanserin binds to other non-serotonergic receptors including adrenergic receptors [78], which may contribute to its positive effects on set-shifting when administered with saline or psilocybin. Similarly, effects of psilocybin on behavioral flexibility may be independent of the 5HT2A receptors as is the case with its anti-anhedonic effects [22].

Differences between psilocybin and DOI

Our findings that DOI has an opposite effect compared to psilocybin highlight significant differences between these drugs and urge caution in generalizing results obtained with DOI to other serotonergic psychedelics. DOI has been used in several studies to investigate the effect of 5HT2A receptor activation and as a readily available proxy for psychedelic function [66, 79, 80]. However, while psilocybin improved flexibility in our set-shifting task, DOI severely impaired performance. It is likely that the effects of DOI on other moderately complex behaviors will differ from those of other psychedelics, despite the assumed similarity between these drugs. Given the complex pharmacological profiles of psychedelics, caution should be used when attempting to generalize results, even if two compounds have similar affinities for specific receptors. Interestingly, two recent studies reported impaired cognitive flexibility in human subjects that received LSD, in contrast with the data on psilocybin [20, 81, 82]. These differences may be due to the subtle pharmacological differences between these drugs: LSD, like DOI, has higher affinity for 5HT2A than 5HT2C, while psilocin has more similar affinity to both receptors; LSD and psilocin both have moderately high affinity at 5HT1A receptors, while DOI does not; and LSD also binds to dopamine D1 and D2 receptors [83–86]. The clinical success of psilocybin may then be due to its very specific pharmacological profile, and may not generalize to other psychedelic compounds.

Increased cognitive flexibility as a therapeutic mechanism for psilocybin

Our data show that an acute dose of psilocybin enhances cognitive flexibility. How could this effect support psilocybin’s therapeutic action? Our hypothesis is that psilocybin transiently counters the cognitive biases commonly found in individuals suffering from psychiatric disorders, particularly MDD. One caveat of our study is that we used healthy rats, not a preclinical model of MDD, so the results should be interpreted cautiously. Nevertheless, cognitive flexibility is associated with effective emotional regulation and general psychological health in healthy adults [87, 88], and cognitive rigidity is a hallmark of depression and its associated perseverative cognitive processes such as rumination, all-or-nothing thinking, and acceptance of maladaptive beliefs about the self and environment [15, 18, 89]. Impaired cognitive flexibility is reported in MDD [17, 90–93] and it has a substantial impact on treatment outcome [94, 95]. Specific deficits in set-shifting are known to be present in patients with MDD [17], and, in preclinical models, antidepressant treatment ameliorates set-shifting impairments [96, 97]. There is also a reported link in humans between increased cognitive flexibility and reduction of depressive symptomatology [89].

Psilocybin’s acute effect on cognitive flexibility may therefore provide a path for countering depressive symptoms. While our data do not address whether psilocybin results in a long-lasting increase in cognitive flexibility, even a transient effect during clinician-assisted intervention could be beneficial given that enhanced cognitive flexibility is advantageous during psychotherapy [98]. Current clinical protocols for psilocybin therapy involve extensive professional support and clinical-guided intervention [9]. We propose that the cognitive state brought about by psilocybin is especially favorable to receive therapeutic benefits from these treatments. This would support the current ideas that the set, the setting and the content of psilocybin administration sessions are all essential to achieve therapeutic benefits: psilocybin, by enhancing cognitive flexibility, may simply allow for a more effective intervention.

Author contributions

RJO, ATP, and BM conceptualized and designed experiments. RJO, GG, and ATP performed experiments. ATP analyzed data. ATP and BM wrote the paper.

Funding

This work was supported by NIMH grants R01MH115027 and R01MH048404. ATP and RJO are supported by NIDA training grant T32DA007262.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41386-023-01545-z.