Antidepressant effects on serotonin 1A/1B receptors in the rat brain using a Gene × Environment model

Abstract

A gene--environment (G×E) interaction is implicated in both the pathophysiology and treatment of major depressive disorder (MDD). This study modeled the effects of genetic vulnerability by using the Flinders Sensitive Line (FSL), a rat model of depression and its control counterpart—the Flinders Resistant Line (FRL). The effects of environmental vulnerability (e.g. early-life stress) were modeled by using maternal separation. Rats (n=105) were drawn from four groups reflecting experimental crossing of strain (FSL vs. FRL) and early-life stress (high vs. low) to assess the effects of two antidepressants (escitalopram or nortriptyline) compared to vehicle. Quantitative in vitro autoradiography was performed using [¹²⁵I]MPPI (5-HT_1a) and [¹²⁵I]CYP (5-HT_1B) in prefrontal cortex (PFC) and hippocampus. Stringent, Bonferroni-corrected statistical analyses showed significant strain-by-rearing-by-treatment (three-way) interactions in PFC 5-HT_1a and hippocampal 5-HT_1B receptors. Either vulnerability reduced serotonergic binding; no additive effects were associated with the two vulnerabilities. Both antidepressants increased hippocampal 5-HT_1B receptor binding; however, only nortriptyline selectively increased PFC 5-HT_1a receptor binding. Taken together, our findings demonstrate that antidepressant effects on the serotonergic system are shaped by a G×E interaction that is dependent on antidepressant class and brain region.

1. Introduction

Selective serotonin reuptake inhibitors (SSRIs) and tricyclic antidepressants (TCAs) are commonly used to treat major depressive disorder (MDD). These antidepressants exert their therapeutic effects via various neurotransmitter systems, including the serotonin (5-HT) system [28]. However, clinical response to antidepressants is highly variablewith only about a third of patients attaining remission [17]. Considerable evidence suggests that this variability in antidepressant response, other than the “placebo effect,” arises from a gene-environment (G×E) interaction; this includes events that unfold during early-life development, and may subsequently influence behavior, treatment outcome, and serotonergic function [10 , 12, 16, 26, 31]. Nevertheless, it is unclear how antidepressants interact with G×E to influence the serotonergic system that may contribute to variability in antidepressant response.

Given the difficulties inherent in conducting such work in humans, rodents pose an excellent and viable alternative. In contrast to clinical studies, which are invariably impacted by uncontrollable environmental and genetic/ethnic differences, preclinical studies using animal models of depression offer several key advantages, including the well-controlled environmental setting and similar genetic background. While a number of suitable rodent models exist, we selected the Flinders Sensitive Line (FSL), a well-established genetic rat model of depression that meets face-, construct-, and predictive-validity criteria [20]. The FSL strain was derived from Sprague-Dawley (SD) rats through a selective breeding program for increased sensitivity to the anticholinesterase compound di-isopropyl fluorophosphates (DFP). Flinders Resistant Line (FRL)—the control counterpart—exhibit low sensitivity to DFP. Notably, several neurochemical and behavioral studies comparing FSL and FRL rats have reported differences in 5-HT levels and neurochemistry, antidepressant effects, and depression-like phenotypes [3, 6, 9, 14, 21-24].

While rodent models such as the FSL capture genetic risk, they also enable investigators to examine the delicate interplay between genetic and environmental vulnerabilities. This is particularly important in the context of MDD because environmental factors affect both pathophysiology and treatment outcome. In particular, adverse early-life events (e.g. childhood maltreatment) increase susceptibility to MDD into adulthood and modulate treatment outcome [2, 26]. In conjunction with genetic or epigenetic factors, the effects of early-life events may alter brain neurochemistry, including that of the serotonergic system [4, 15].

Both animal and human studies have implicated the abnormal functions of 5-HT_1a and 5-HT_1B receptors in the pathophysiology of MDD and demonstrated that biochemical adaptations occur in both receptor subtypes with antidepressant treatment [25, 27]. Both receptor subtypes—expressed in prefrontal cortex (PFC) and hippocampus—regulate 5-HT firing as well as aid in serotonergic neurotransmission due to their presence both pre- and post-synaptically. Pre-synaptically, they regulate neuronal firing and 5-HT release via a negative-feedback system. Post-synaptically they facilitate neurotransmission, which ultimately affects a myriad of physiological processes such as mood, sex, and appetite. The delayed therapeutic onset of SSRIs is postulated to the time necessary for the desensitization of somatodendritic5-HT_1a autoreceptors in raphe [1].

The present study investigated the neurochemical effects of two antidepressants from different classes—escitalopram (an SSRI) and nortriptyline (a TCA)—in a rat model with genetic (FRL vs. FSL) and/or environmental (normally-reared vs. maternally-separated rats) vulnerabilities. Quantitative in vitro autoradiography was performed to examine the interactions between 5-HT_1a and 5-HT_1b receptors in both PFC and hippocampus. The study had three primary hypotheses: 1) both genetic vulnerability (modeled by rat strain) and environmental vulnerability (modeled by maternal separation) affect the serotonergic neurochemistry; 2) a G×E interaction affects serotonergic neurochemistry; and 3) antidepressants would selectively modulate serotonergic neurochemistry in a G×E interaction manner. The three hypotheses were tested within the context of an omnibus statistical model – i.e., a strain-by-stress-by-treatment (three-way) interaction.

2. Material and Methods

2.1 Study Design

The three-way interaction (2 × 2 × 3 matrix) on the serotonergic system was examined as shown in Figure 1.

Fig. 1.

A three-way study design examining strain by stress by treatment (2 × 2 × 3 matrix) interaction on the serotonergic system. In total, 105 rats were divided across 12 experimental groups.

2.2 Maternal Separation and Antidepressant Treatment

Maternal separation and antidepressant treatment were performed as previously described [6, 21, 22]. Briefly, the study used only male rats housed in pairs in an 1800 cm² cage under a 12-hour light/dark reverse cycle at 21°C, relative humidity 55%, and food and water ad libitum. Post-natal day 0 (PND0) was designated as the day of birth. To model early-life stress, half of the pups were separated from their mothers for 180 min/day for 12 days starting at PND2.

On PND43, equal number of rats in each group was randomly assigned to 30-day dietary supplementation (diet prepared by Lactamin AB, Sweden) with either escitalopram (0.34 g/kg for the first three weeks, 0.41 g/kg during the rest of the experiment) or nortriptyline (0.34 g/kg), or vehicle. The daily administered dose was 25 mg/kg for escitalopram and 20 mg/kg for nortriptyline. Antidepressant serum concentrations were not measured; however, several prior studies found that the selected oral doses provide serum concentrations of 15-35 ng/mL (AAM obtained proprietary results in collaboration with Lundbeck A/S and Pfizer), which are typically used to model therapeutic concentrations in humans. Average food pellet intake during the treatment period was 22 g/rat/day. No difference in food consumption or total brain weight was observed between genotypes. However, a significant weight difference emerged between the strains (FRL = 209±2.4 vs. FSL = 196±2.0).

At the end of the study rats were sacrificed, and brains were immediately harvested and frozen at − 80 °C. Brain was sectioned (14 μm) using a microtome-cryostat (−20 °C) and mounted on gelatin-coated slides. The Stockholm Ethical Committee for the Protection of Animals approved the study.

2.3 Drugs and radioligands

[¹²⁵I]MPPI and [¹²⁵I]-(±)cyanopindolol were purchased from PerkinElmer (Massachusetts, USA). 5-HT, 8-OH-DPAT, isoprenaline, and all other reagents used in the study were purchased from Sigma (Sweden).

2.4 [¹²⁵I] autoradiography

[¹²⁵I] autoradiography was performed as previously described with minor modifications [13, 30] in brain sections of the PFC and hippocampus. All sections were rinsed (30 min) with 50 mM Tris-HCl at 24 °C. For the 5-HT_1a receptor, sections were pre-incubated (30 min) in binding buffer (50 mM Tris-HCl and 2 mM MgCl₂, pH 7.5) followed by two-hour incubation with antagonist [¹²⁵I]MPPI (120 pM; 2200 Ci/mmol) at 24 °C. Non-specific binding was determined in the presence of 10 μM 8-OH-DPAT. Sections were washed 2 × 15 min with ice-cold binding buffer. For the 5-HT_1b receptor, sections were pre-incubated (30 min) at 24 °C in binding buffer (170 mM Tris-HCl and 150 mM NaCl, pH 7.5) followed by two-hour incubation with antagonist [¹²⁵I]cyanopindolol (12 pM; 2200 Ci/mmol), 30 μM isoprenaline, and 100 nM 8-OH-DPAT to block the β-adrenergic receptor and 5-HT_1a receptor, respectively. Non-specific binding was determined in the presence of 10 μM 5-HT. Sections were washed 2 × 5 min with ice-cold binding buffer. All sections were quickly dipped in ice-cold distilled water, dried, and exposed, together with ¹²⁵I plastic microscale standards (American Radiolabeled Chemicals, Missouri, USA), to Kodak BioMax MR films for 24 hours ([¹²⁵I]MPPI) or 48 hours ([¹²⁵I]cyanopindolol) at 4 °C.

2.5 Data analysis

Autoradiograms were quantified using NIH ImageJ (1.44p) (http://rsbweb.nih.gov/ij/). Optical densities were normalized per area and converted into fmol/mg tissue based on the microscale standards by using nonlinear regression fit (GraphPAD Prism). Specific binding was determined by subtracting non-specific from total binding. In all graphs, the control group was normally-reared, vehicle-administered FRL rats. All receptor binding data are expressed as percent of control calculated as: receptor density *100/average receptor density of control group. All data are presented as % of control ± SEM.

2.6 Statistics

For each of the two regions and two radioligands analyzed, a full factorial ANOVA was used to examine a possible strain-by-stress-by-treatment interaction. Bonferroni post-hoc tests were used to evaluate omnibus main effects and interactions. In addition, a priori comparisons were used to examine G×E interactions within the vehicle group before examining treatment effect relative to vehicle. After Bonferroni correction for multiple comparisons, the cutoff p-value was < 0.0125. All analyses were run using IBM SPSS 19.0.0.1 (http://www-01.ibm.com/software/analytics/spss/).

3. Results

After stringent corrections for Bonferroni and multiple regions, significant strain-by-rearing-by-treatment (three-way) interactions emerged for only two of the four dependent measures: 5-HT_1a receptors in the PFC and 5-HT_1b receptors in the hippocampus.

5-HT_1a receptors

[¹²⁵I]MPPI autoradiograms showed specific binding in PFC and hippocampus (Fig. 2A & 2B).

Fig. 2.

Autoradiograms of 5-HT_1a receptors using [¹²⁵I]MPPI in (A) PFC and (B) dorsal hippocampus; (C) A three-way interaction emerged in the PFC (F=5.55, df=2,75, p= 0.006). Nortripytline normalized the combined effects of G×E.

In the PFC, a significant strain-by-rearing-by-treatment (three-way) interaction emerged (F=5.55, df=2,75, p< 0.01) (Fig. 2C; Table 1). Initial post-hoc analyses showed that both vulnerable genotype and environment reduced 5-HT_1a receptor binding (p< 0.05). However, the effects of these two vulnerabilities were not additive. A second set of post-hoc analyses showed that only nortriptyline selectively increased 5-HT_1a receptors in the group with both vulnerabilities (p< 0.001).

Table 1.

Antidepressant effects on the serotonergic system in rat prefrontal cortex (PFC) and hippocampus (HP) using a gene by environment (GxE) model.

Vulnerability		Antidepressant effects
Genotype	Early-life stress	Escitalopram	Nortriptyline
FRL (low)	normally reared (low)	no change compared to vehicle
FRL (low)	maternally separated (high)	↑ 5-HT1b (HP)*	↑ 5-HT1b (HP)*
FSL (high)	normally reared (low)	↑ 5-HT1b (HP)*	↑ 5-HT1b (HP)*
FSL (high)	maternally separated (high)	↑ 5-HT1b (HP)**	↑ 5-HT1a (PFC)*

^*p< 0.001;

^**p< 0.05 (Bonferroni post-hoc correction and correction for multiple regions)

FRL: Flinders Resistant Line; FSL: Flinders Sensitive Line

In the hippocampus, significant rearing-by-treatment (F=6.32, df=2,76, p< 0.01) and G×E (F=11.55, df=2,76, p< 0.01) interactions emerged. Both antidepressants increased 5-HT_1a receptors in the maternally-separated group.

5-HT_1b receptors

[¹²⁵I]cyanopindolol autoradiograms showed specific binding in both PFC and hippocampus (Fig. 3A & 3B).

Fig. 3.

Autoradiograms of 5-HT_1b receptors using [¹²⁵I]cyanopindolol in (A) PFC and (B) dorsal hippocampus; (C) A three-way interaction emerged for hippocampus (F=8.30, df=2,75, p< 0.001). Escitalopram normalized the combined effects of G×E, while nortriptyline normalized the effect of either strain or stress.

In the PFC, a significant rearing-by-treatment interaction emerged (F=12.31, df = 2,77, p< 0.001) (Fig 3C; Table 1). In the maternally-separated group, escitalopram increased 5-HT_1b receptor binding (p < 0.05).

In the hippocampus, a significant strain-by-rearing-by-treatment (three-way) interaction emerged (F=8.30, df=2,75, p< 0.001) (Fig. 3C). Initial post-hoc analyses revealed that both vulnerable genotype and environment reduced hippocampal 5-HT_1b receptor binding (p< 0.05). Similar to 5-HT_1a receptors, the effects of these two vulnerabilities were not additive. Escitalopram increased binding in the group with either or both vulnerabilities (p< 0.01). In contrast, nortriptyline increased binding in the group only with either vulnerability (p< 0.001).

4. Discussion

This study investigated the effects of two different classes of antidepressants on the serotonergic neurochemistry in rat brain using a G×E model. Antidepressant effects on 5-HT_1A/1B receptors were shaped by complex G×E interactions. Specifically, 1) both vulnerable genotype (FSL strain) and vulnerable environment (early-life stress) reduced 5- HT_1A/1B receptor binding; 2) the effects of genotype and environment were not additive; and 3) antidepressants, in general, increased 5- HT_1A/1B receptor binding (Table 1).

With regard to the vehicle-treated rats, binding for each receptor was found to be generally lower in groups with either genetic or environmental vulnerability. However, no additive effects were associated with the two vulnerabilities, suggesting that altered serotonergic functioning stemming from either vulnerable genes or a vulnerable environment can manifest into depression/anxiety-like phenotypes. This echoes prior findings demonstrating that either early-life stress or genetic manipulations that reduced the number of 5-HT_1a and/or 5-HT_1b receptors in rodents could result in depressive/anxiety-like phenotypes [8, 29, 30].

As regards treatment effects, unique three-way G×E interactions specific to antidepressant class emerged for PFC 5-HT_1a receptors and hippocampal 5-HT_1b receptors. Both antidepressants, in general, increased serotonergic binding. Escitalopram increased hippocampal 5-HT_1b receptor binding in groups with either or both vulnerabilities. In contrast, nortriptyline increased hippocampal 5-HT_1b receptor binding in groups with either, but not both, vulnerabilities; however, nortriptyline selectively increased PFC 5-HT_1a receptor binding only in the group with both vulnerabilities (Table 1).

In rodents, previous neurochemical studies using similar paradigms that focused on either—but not both—genetic or environmental vulnerabilities reported altered 5-HT_1A/1B receptor binding, although findings are inconsistent [7, 19, 30]. This discrepancy may arise from several sources, including rat strain, methodology (e.g., autoradiography vs. membrane homogenate binding), brain region, duration of treatment, age, and sex. Future studies that control for as many variables as possible are necessary to clearly elucidate their interplay.

Interestingly, behavioral studies in the Flinders strain found that either genetic or environmental vulnerabilities could result in depressive-like phenotypes, with complex interactions between genotype, environment, and antidepressant response [6, 21-24]. For instance, studies found increased immobility in FSL vs. FRL rats, and maternal separation was found to increased immobility more in FSL than FRL rats [6, 22, 24]. Chronic escitalopram decreased immobility only in normally-reared rats of either strain [24]. On the other hand, chronic nortriptyline decreased immobility only in FSL rats, regardless of rearing [21, 23].

In humans, a similar G×E interaction is currently favored for explaining response variability. Though the topic is presently being debated, several studies suggest that genetic polymorphisms (e.g. 5HTTLPR) and early-life stressors (e.g. physical abuse) interact to influence mood and antidepressant response [4, 10, 12, 16, 26]. The most influential G×E study suggests that 5HTTLPR—a polymorphism in the promoter region of the serotonin transporter gene (SLC6A4)—moderates the relationship between environmental stress and mood disorders [4]. In particular, this study reported a positive relationship between the number of self-reported early-life stressors and risk for depression among individuals with one or two copies of the short allele compared with those homozygous for the long allele. However, more than 50 replication attempts, as well as three meta-analyses, have yielded inconsistent findings [5]. The inconsistency is attributable to publication bias towards positive findings, which may be statistically underpowered and susceptible to false positives [11].

Our proof-of-concept study mirrors key aspects of the clinical scenario, in the sense that genes, environment, and antidepressant class were explicitly manipulated. Each of these factors is known to contribute to treatment outcome. The experiment used a relatively large number of animals across the entire interaction study; however, the presence of three experimental factors yielded a design where each group contained relatively few animals, which is susceptible to false positives [5]. However, based on emerging—albeit inconsistently applied—standards [18], we adopted a rigorous statistical approach, with Bonferroni correction of omnibus three-way interactions.

5. Conclusions

In conclusion, significant strain-by-rearing-by-treatment (three-way) interactions emerged for 5-HT_1a receptors in the PFC and 5-HT_1b receptors in the hippocampus. The findings demonstrate interactions of at least three known variables on the serotonergic neurochemistry where antidepressant class modulates the delicate interplay between genes and environment in a brain region-dependent manner. Our findings provide preliminary validity for using this controlled animal model, albeit with larger sample sizes, to further understand the interaction among these three key factors in MDD: genes, environment, and antidepressant class.

Highlights.

• Complex gene/environment interactions shaped how antidepressants affect 5-HT_1A/1B receptors.

• Strain-by-rearing-by-treatment interactions were observed for PFC 5-HT_1a receptors.

• Strain-by-rearing-by-treatment interactions were observed for hippocampal 5-HT_1b receptors.

• Either genetic or environmental vulnerability reduced 5-HT_1A/1B receptor binding.

• The effects of genotype and environment were not additive.