Strain and sex based characterization of behavioral expressions in non-induced compulsive-like mice

Abstract

There is currently a lack of understanding how genetic background and sex differences attribute to the heterogeneity of obsessive-compulsive disorder (OCD). An animal model of compulsive-like behaviors has been developed through bidirectional selection of house mice (Mus musculus) for high (big cotton nests; BIG mice) and low levels (small nests; SMALL mice) of nest-building behavior. The BIG male strains have predictive and face validity as a spontaneous animal model of OCD. Here, we evaluated compulsive-, anxiety-, cognitive-, and depression-like behaviors among male and proestrus female replicate strains each of BIG (BIG1, BIG2) and SMALL (SML1, SML2) nest-builders, and randomly-bred Controls (C1, C2). BIG1 and BIG2 males and females had higher nesting scores when compared to SMALL and Control strains. Male BIG1 and BIG2 strains showed more compulsive-like nesting than BIG1 and BIG2 proestrus females, which was not observed among the other strains. Nesting scores were also different between BIG replicate male strains. A similar pattern was observed in the compulsive-like marble burying behavior with BIG strains burying more marbles than SMALL and Control strains. Significant replicate and sex differences were also observed in marble burying among the BIG strains. The open field test revealed replicate effects while the BIG strains showed less anxiety-like behavior in the elevated plus maze test compared to the SMALL strains. For novel object recognition only the Control strains showed replicate and sex differences. In the depression-like forced swim test proestrus females demonstrated less depression-like behavior than males. BIG and SMALL nest-building strains had a higher corticosterone stress response than the Control strains. Together these results indicate a strong interplay of genetic background and sex in influencing expression of behaviors in our compulsive-like mouse model. These results are in congruence with the clinical heterogeneity of OCD.

1. Introduction

Obsessive-compulsive disorder (OCD) is a debilitating psychiatric condition characterized by invasive and persistent thoughts (obsessions) and repetitive behaviors (compulsions) [4]. OCD has an estimated lifetime prevalence of 2.3% in the United States [97]. The majority of patients suffering from OCD perform repetitive rituals to mitigate uncomfortable feelings of anxiety [32]. Others engage in repetitive behaviors due to subjective sensations also called sensory phenomena [79]. Obsessions can be associated with contamination, fear and symmetry, while compulsions include hand washing, checking or counting [88]. Such excessive ritualistic behaviors become distressing and significantly interfere with daily functioning [4]. OCD exhibits a large behavioral repertoire with high rates of repetition and provides an ethological basis for studying compulsive-like behaviors in animal models of OCD [11]. Though inappropriate for investigating the entire OCD spectrum because obsessions cannot readily be assessed, animal models can provide deep insight into various forms of compulsivity [74].

Sex differences in OCD [14, 29, 63] is thought to contribute to the heterogeneity of OCD [59]. Age of onset [14], phenomenology [59] and co-morbidity [69] are some of the gender related differences that have been observed. Males usually have an earlier onset for most symptom dimensions of OCD than females [23, 72, 82, 83]. Clinical studies have indicated that sexual, religious and symmetrical obsessions, and compulsive checking and ordering/arranging are more frequently seen in males than females, while females tend to exhibit more obsessions for contamination and compulsive cleaning than men [30, 62, 72, 108].

In females, ovarian hormones may play a critical role in modulating obsessions and compulsions [113]. Fluctuations in the female hormonal cycle may contribute to increased risk of onset and exacerbation of OCD symptoms at certain reproductive events, including premenstruum phase, menstrual cycle, pregnancy, and postpartum [1, 63, 73, 109, 113]. Few rodent studies have established that the female sex hormone estrogen can modulate compulsive-like behaviors [33, 37, 58]. The proestrus stage of the estrous cycle in female mice has higher circulating levels of estrogen influencing anxiety-, cognitive- and depression-like behaviors [106, 116] and, therefore, the comparison between males and proestrus females for compulsive-like behaviors may provide additional ways to gain important insights using animal models of OCD.

Additionally, inbred female mouse strains have shown significant differences in expression of drug induced compulsive-like behavior [27], indicating that strain comparisons can be valuable for understanding expression of compulsivity. This is important considering the compelling genetic basis of OCD from various human studies [25, 51, 52, 56, 76, 96, 105], which coupled with the sex differences can provide crucial clues about the complex interactions of these two elements in expression of compulsive and affective behaviors.

According to Maio et al. [74], comparing our compulsive-like mouse strains is valuable for understanding the disorder. Compulsive-like behavior in our mice is defined as excessive and repetitive expression of otherwise normal behaviors, i.e., nest-building and marble burying. For example, nest-building in the compulsive-like strains (BIG1 and BIG2) involves rapid and repeated movements of the front legs and mouth to pull excessive amount of cotton through the cage top metal bars over extended periods of time [50], which shows face validity with repetitive behaviors in OCD [79]. This rapid, excessive and repetitive nesting behavior is not observed in the Control and SMALL strains. The BIG strains also exhibit predictive validity as a spontaneous non-induced model of OCD-like behaviors through attenuation of these compulsive-like behaviors with fluoxetine and clomipramine treatment, which are first line treatments for OCD [50]. In addition, the tricyclic antidepressant desipramine, which is not effective in treating OCD, did not change the compulsive-like behaviors in the compulsive-like mice [50]. These strains were developed by bidirectional selection for high and low levels of nest-building behavior [19, 71] using a stock population, i.e., HS/Ibg outbred strain, that was derived from a cross among eight inbred house mouse, Mus musculus, strains, i.e., A, AKR, BLB/c, C3H/2, C57BL, DBA/2, Is/Bi, and RIII [71, 77].

Bidirectional selection resulted in three levels of nest-building behavior (with two replicate strains within each level). The replicates within each level of nest building were maintained as separate strains, i.e., not interbred with the other replicate, but subjected to the same selection regime. Using replicate strains is important to make sure that responses to artificial selection are due to selection and not the result of founder effects or random genetic drift when comparing the different levels of selection, i.e., selection for building big nests or small nests, or randomly bred [17, 19]. The two BIG strains consistently display high levels of nesting with a forty-fold difference in the amount of cotton used when compared to the two SMALL strains which display very low levels of nesting [19, 71]. The two randomly-bred strains serve as a selection Control and show intermediate levels of nesting [19, 71].

Compulsive-like nesting in our mice has a genetic factor with about 30% of the variation in behavioral expression among individuals due to additive genetic factors and about 70% of the variation due to environmental factors [19, 71]. This estimate of heritability of nest building falls within the range of heritability estimates of 0.36 [111], 0.37 [26], 0.23-0.58 [60], and similar estimates for different types of OCD [16]. In addition, quantitative genetic analyses have revealed nesting behavior to be a highly polygenic trait [19, 71], which is consistent with OCD in humans likely being influenced by many different genes [57, 89, 101].

Considering the heterogeneity and sex factors in OCD and the behavioral and genetic characteristics of our mouse model of OCD, we aimed to investigate the effects of genetic background and sex on the compulsive-, anxiety-, cognitive- and depression-like behaviors for a better understanding of factors influencing compulsivity.

2. Methods

The University of Alaska Fairbanks Institutional Animal Care and Use Committee approved the animal care and experimental procedures (IACUC assurance number 497513).

2.1. Mice

2.1.1. Husbandry

Mice were housed in polypropylene cages (27×17×12 cm) with wood shavings in a temperature (22±1°C) and humidity (60-80%) controlled room of the Biological Research and Diagnostics (BiRD) Facility vivarium on a 12:12h light-dark cycle. Pups were weaned at 19-21 days of age and housed with same-sex and same-strain littermates until the end of all experiments. Food (Purina Mills, Lab Diet Mouse Diet #5015, St. Louis, MO) and water were available ad libitum.

2.1.2. Mouse strains

Randomly-bred Control strains (C1, C2), BIG strains selected to build big nests (BIG1, BIG2) and SMALL strains selected to build small nests (SML1, SML2) [17, 18, 19] were used. The BIG strains express higher levels of compulsive-like behaviors, i.e., nest building and marble burying, when compared to the SMALL strains [50], while the Control strains are intermediate [19, 71]. Male and female mice (Mus musculus) of these six different mouse strains were used for the behavioral experiments (C1 males n=20; C1 females n=21; C2 males n=20; C2 females n=17; BIG1 males n=21; BIG1 females n=22; BIG2 males n=20; BIG2 females n=14; SML1 males n=19; SML1 females n=21; SML2 males n=19; SML2 females n=20).

2.1.3. Estrous cycle

Female mice in proestrus were used for the study. The proestrus stage was determined daily by both visual and vaginal cytology methods [20]. All females from all the strains were visually inspected daily. Vaginal cytology was performed on those females that showed visual signs of proestrus. The females for which proestrus was confirmed through cytology were subjected to behavioral testing the same day. This procedure was conducted daily until desired sample sizes were achieved for a specific test. When the females cycled to their next proestrus, which was typically on the 4th or the 5th day, they were subjected to the next behavioral test in the schedule and this cycle was continued until all behavioral tests were completed.

2.2. Experimental design

All mice were at least 60 days old at the start of the experiment. All behavioral tests were done in the light phase of the light-dark cycle. All males underwent behavioral tests once in every 5 days consisting of nest building (on day 1), marble burying (on day 6), open field (on day 11), elevated plus maze (on day 16), novel object recognition (on day 21) and forced swim test (on day 26). For females, behavioral tests were conducted every 4-5 days depending on their proestrus stage. Males and females were tested on this schedule until desired sample sizes were achieved. A gap of 4 days for males and 3-4 days for females between each behavioral test minimized the behaviors interfering with each other.

For nest-building, animals were housed individually in cages and, therefore, following data collection after 24 hours, they were returned to home cages with their littermates and kept for 4 days to negate any isolation effects that could influence performance in subsequent tests [55]. For marble burying, open field, elevated plus maze, and novel object recognition the mice were singly housed just before testing. After testing they were returned to their home cage with their littermates. Upon reintroduction to their littermates, some fighting was observed within the first hour, which was similar to fighting observed after a normal cage change, which should not have interfered with the subsequent behavioral assessment considering a 3-4 day gap between tests. Immediately after the forced swim test, trunk blood was collected to determine stress-induced plasma corticosterone levels. An observer blind to the conditions of experimental animals and the hypothesized outcome of the study collected all data.

2.3. Measuring compulsive-like behavior

2.3.1. Nest-building test

Nest-building behavior was used as a measure of the compulsive-like phenotype of the mice [50]. Male and female mice of all the strains were singly housed and provided with a pre-weighed cotton roll in the cage-top food hopper. After 24 hours, the cotton roll was removed and weighed [17, 18, 19, 71]. The total nesting score was defined as the amount of cotton used over the 24-hour period. Generally, all the mice incorporated the cotton into their nest. The more cotton was used, the more elaborate the nest was, which progressed from cotton on the bottom of the nest, to a bowl nest, and finally to a dome nest.

2.3.2. Marble burying test

Compulsive-like behavior was also assessed through the marble burying test [2, 7, 31, 50]. Mice were placed in a standard housing cage with 5 cm deep clean bedding and 9 glass marbles arranged in two rows. Testing was performed in the presence of white noise. The total number of marbles 2/3 buried was quantified. The current protocol was modified from existing protocols in our lab [50] and other studies [24, 31, 107]. The mouse strains were found to replicate the 20-minute protocol with 20 marbles in a 10-minute trial with 9 marbles. Our group has also published a study with rats where 9 marbles for 10 minutes provided conclusive findings [70].

2.4. Measuring anxiety-like behaviors

2.4.1. Open field test

Open field behavior was used to assess anxiety-like behavior of the mice [92]. BIG, SMALL and Control male and female mice were transported in their home cages, singly housed and placed on a rack outside the testing room just prior to testing. The open field apparatus consisted of an open field arena (40 × 40 × 30 cm) with 16 10×10 cm squares marked on its floor. For testing, mice were placed in the central 4 squares of the field and allowed to explore the arena for 5 minutes. Entries into the central (20 × 20 cm) and peripheral squares were recorded [40] by the ANYMaze video-tracking program (Stoelting Co., IL, USA). The apparatus was cleaned before each test.

2.4.2. Elevated plus maze

Assessment of anxiety-like behavior was also determined with the elevated plus maze test [114]. The plus maze consisted of two open arms (5 × 40 cm) and two closed arms (5 × 40 × 20 cm) at right angles to each other. Each mouse was placed in the central square facing an open arm, and was allowed to explore the maze for 5 minutes. The time spent on the open arms was determined [115] by the ANYMaze video tracking program (Stoelting Co., IL, USA). An entry was defined as all four paws being on the arm. The maze was cleaned before each test.

2.5. Measuring novel object recognition behavior

The novel object recognition test was performed to measure object recognition memory [8]. During day 1, mice were habituated to the open field arena (40 × 40 × 30 cm) for 3 minutes. Twenty-four hours later on day 2, the time spent on investigating two identical objects (plastic toys) within a 5 cm distance in the open field arena was recorded for 3 minutes with the ANYMaze video tracking program (Stoelting.co). Mice were then taken out of the arena and returned to their home cages for 4 hrs. After 4 hrs one of the objects was replaced with a novel object of different shape and size and animals were then reintroduced into the arena and allowed to explore the objects for 3 minutes. Time spent exploring the familiar and novel objects were recorded. The preference of one object over another was assessed through the Recognition Index (RI) which is the time spent on the novel object relative to the time spent on both novel and familiar objects: [RI = T_N/(T_N + T_F)] where T_N is time spent on the novel object and T_F is time spent on the familiar object) [38, 67].

2.6. Measuring depression-like behavior

Time spent immobile in the forced swim test was used as a measure of depression-like behavior [41, 90, 91], where immobility is defined as the absence of no active behaviors like swimming, jumping or diving [12, 41, 91]. Mice were introduced in a glass cylinder, 20.5 cm in diameter and 21.5 cm in depth, which was filled with 18 cm of water maintained at 25 - 27°C for 10 minutes [44].

2.7. Corticosterone ELISA

Mice were euthanized after the forced swim test through cervical dislocation to measure stress induced corticosterone levels. Trunk blood was collected randomly from each strain and sex (C1 males n=4; C1 females n=5; C2 males n=6; C2 females n=6; BIG1 males n=4; BIG1 females n=4; BIG2 males n=7; BIG2 females n=5; SML1 males n=5; SML1 females n=5; SML2 males n=6; SML2 females n=6) in chilled heparinized tubes and stored at -80°C. After thawing the blood was centrifuged at 1000 × g for 20 minutes. The plasma was extracted and placed in fresh tubes and stored at −80°C. The corticosterone levels were determined using a corticosterone ELISA kit from Cayman Chemicals (Catalogue ID: 500655) as per the manufacturer's instructions and as previously described [42]. The detection limit (80% B/B₀) of the assay is approximately 80 pg/mL (Cayman) with inter- and intra-assay coefficient of variation of 2.2% and 8.9%, respectively.

2.8. Statistical analyses

All measurement values are expressed in mean ± standard error of the mean (SEM). Statistical Analysis System (SAS Version 9.4, Cary, NC) software was used for statistical analyses. All behavioral and biochemical measures were tested in a general linear model (GLM) analysis of variance (ANOVA) for effects of strain (BIG, SMALL, Control), sex (female, males), replicate (1, 2) nested within strain, and strain × sex interaction. If significant effects were found, appropriate post-hoc pair-wise comparisons were conducted using the Tukey's Studentized Range test. If the replicate effect was significant, the strain effect was tested over the replicate effect. If the replicate effect was not significant, the strain effect was tested over the error term. The total nesting score was square root transformed to obtain a more normal distribution [17, 18, 19], while the data are presented as non-transformed nesting scores. Significant correlations between behaviors within strains were determined through linear regression.

3. Results

3.1. Behavioral Data

3.1.1. Strain, replicate and sex differences in nest building

Significant strain (F_2,3=175.41, p<0.001), replicate (F_3,226=3.71, p<0.02) and sex (F_1,226=15.61, p<0.0001) effects were observed for compulsive-like nest building. Both male and female big nest builders (BIG1 and BIG2) used more cotton to build a nest than the Control mice (C1 and C2) and the SMALL mice (SML1 and SML2), and the Control mice built bigger nests than the SMALL mice. Most SML2 male mice did not use any cotton. The replicate effect was due to significant differences between the BIG1 and BIG2 males and C1 and C2 females. Sex differences were attributed to BIG1 and BIG2 males using more cotton than proestrus females for nesting (Fig. 1). The significant interaction between strain and sex (F_2,226=16.68, p<0.0001) was due to females and males not being different in total nesting scores in the C1 and C2, and SML1 and SML2 strains, while the BIG strains showed large differences.

Figure 1. Nest-building behavior.

Mean (± SEM) total nesting scores of male and female BIG, SMALL and Control replicate strains. BIG1 and BIG2 males had higher nesting score than C1 (t=15.28 p<0.001 and t=12.95 p<0.001), C2 (t=17.20 p<0.001 and t=14.86 p<0.001), SML1 (t=20.47 p<0.001 and t=18.16 p<0.001) and SML2 (t=22.72 p<0.001 and t=20.42 p<0.001) males. BIG1 males had higher nesting score than BIG2 males (t=2.365 p<0.05). BIG1 and BIG2 females had higher nesting score than C1 (t=9.564 p<0.001 and t=9.364 p<0.001), C2 (t=11.81 p<0.001 and t=11.43 p<0.001), SML1 (t=15.19 p<0.001 and t=14.34 p<0.001) and SML2 (t=14.20 p<0.001 and t= 13.49 p<0.001) females. BIG1 and BIG2 males had higher nesting score than BIG1 and BIG2 females (t=6.682 p<0.001 and t=2.886 p<0.05).

3.1.2. Replicate and sex differences in marble burying

For compulsive-like marble burying, the strain differences were marginally significant (F_2,3=9.02, p<0.054), predominantly due to the significant replicate effect (F_3,226=10.21, p<0.0001), although the general trend of BIG mice burying more marbles than SMALL mice with Control mice having intermediate values was in line with previously found significant strain differences [53]. The replicate effect was predominantly due to C1 females burying more marbles than C2 females. The significant sex effect (F_1,226=11.03, p<0.002) resulted from C2 and BIG1 males burying more marbles than their female counterparts (Fig. 2). The significant sex by strain interaction effect (F_2,226=4.07, p<0.02) was due to the SMALL strains not showing sex or replicate effects while the other strains did.

Figure 2. Marble burying behavior.

Mean (± SEM) number of marbles buried in male and female BIG, SMALL and Control replicate strains. Replicate effect was due to C1 females burying more marbles than C2 females (t=5.850 p<0.001). BIG1 and C2 males buried more marbles than BIG1 (t=3.132 p<0.05) and C2 (t=4.201 p<0.001) females, contributing to the sex differences and strain by sex interactions.

3.1.3. Replicate differences in open field behavior

There were no significant strain (F_2,3=1.16, p>0.40), sex (F_1,226=1.73, p>0.15) and strain by sex interaction (F_2,226=2.54, p= 0.08) effects on anxiety-like behavior in the open field, as measured by the number of central entries. Significant replicate effects (F_3,226=17.87, p<0.0001) were due to female and male C2 mice making more central entries than C1 mice. Replicate effects were also due to female and male SML1 mice making more central entries than SML2 mice (Fig. 3).

Figure 3. Open field behavior.

Mean (± SEM) number of central entries in male and female BIG, SMALL and Control replicate strains. C2 males and females had more central entries than C1 males (t=3.739 p<0.001) and females (t=3.903 p<0.001), respectively. SML1 males and females had more central entries than SML2 males (t=3.624 p<0.001) and females (t=3.903 p<0.01), respectively.

3.1.4 Strain differences in elevated plus maze behavior

In the elevated plus maze test, anxiety-like behavior, as measured by the time spent on the open arms, was significantly influenced by strain (F_2,226=22.11, p<0.0001) without a significant replicate effect (F_3,226=1.11, p>0.30). The strain differences were due to BIG2 males spending more time on the open arms than C2, SML1 and SML2 males and the BIG1 and C1 males spending more time on the open arms than the SML2 males. For females, the C1, C2, SML1 and SML2 strains spent less time on open arms than the BIG1 and BIG2 females. No significant sex (F_1,226=0.75, p>0.30) and strain by sex interaction (F_2,226=0.74, p>0.40) effects were found (Fig. 4).

Figure 4. Elevated plus maze behavior.

Mean (± SEM) time spent on open arms of male and female BIG, SMALL and Control replicate strains. Significant strain differences were observed in time spent on open arms between male BIG2 with C2 (t=3.881, p<0.001), SML1 (t=3.807, p<0.001) and SML2 strains (t=4.899, p<0.001). The C1, C2, SML1 and SML2 females spent less time on open arms when compared to BIG1 females (t= 3.836, p<0.001; t=2.627, p<0.05; t=4.041, p<0.001; t=3.923, p<0.001, respectively). The C1, SML1 and SML2 females spent less time on open arms when compared to BIG2 females (t= 2.904, p<0.01; t=3.085, p<0.01; t=2.994, p<0.01, respectively).

3.1.5 Sex and replicate differences in novel-object recognition memory

No significant strain (F_2,3=5.33, p>0.09) and strain by sex interaction (F_2,226=2.80, p>0.05) effects were found for novel object recognition. The significant effects of sex (F_1,226=6.87, p<0.01) on novel object recognition was mostly due to female C1 mice showing better performance than their male counterparts (Fig. 5). A significant replicate effect was found (F_3,226=3.44, p<0.01), which was predominantly due to differences between C1 and C2 males.

Figure 5. Novel Object Recognition behavior.

Mean (± SEM) recognition index of male and female BIG, SMALL and Control replicate strains. The C2 males had higher RI values when compared to C1 males (t=3.340, p<0.01). The C1 females had higher RI when compared to C1 males (t=3.921, p<0.001).

3.1.6 Sex and replicate differences in the forced swim test

In the forced swim test, depression-like behavior, as measured by duration of immobility, revealed no significant strain (F_2,3=1.78, p>0.25) and strain by sex interaction (F_2,226=1.82, p>0.15) effects. The significant sex (F_1,226=5.67, p<0.05) effect was due to proestrus females demonstrating less depression-like immobility behavior than did males, especially SML1, BIG1, C1, but not SML2 strains (Fig. 6). The significant replicate effect (F_3,226=8.64, p<0.0001) was due to differences between the SML1 and SML2 males and the C1 and C2 females.

Figure 6. Forced Swim Test.

Mean (± SEM) immobility time of male and female BIG, SMALL and Control replicate strains. SML1 males had higher immobility times than SML2 males (t=5.796, p<0.001) while C2 females had higher immobility times than C1 females (t=4.229, p<0.001). BIG1, C1 and SML1 males had higher immobility times when compared to BIG1 (t=3.063, p<0.05), C1 (t=3.778, p<0.01) and SML1 (t=3.083, p<0.01) females. In the SML2 strain, females had higher immobility time than the males (t=3.827, p<0.01).

3.2. Strain differences in corticosterone plasma levels

Plasma corticosterone levels were significantly influenced by strain (F_2,52=12.17, p<0.0001) without a replicate effect (F_3,52=2.53, p>0.05). The BIG and SMALL nest-building strains generally mounted a higher corticosterone response to forced swim followed by euthanasia. Corticosterone levels in male BIG1 and SML1 strains were significantly higher than the C2 strain (Fig. 7). Also, BIG1 males had higher corticosterone levels than C1 males. In females, both the BIG and SMALL strains had higher corticosterone levels when compared to the Control strains. No significant sex (F_3,52=3.68, p>0.05) and strain by sex interaction (F_2,52=1.54, p>0.20) effects were observed.

Figure 7. Plasma Corticosterone Levels.

Mean (± SEM) corticosterone levels as a measure of HPA axis stress response of male and female BIG, SMALL and Control replicate strains. Levels in male BIG1 and SML1 strains were significantly higher than the Control C2 strain (t=3.773, p<0.001 and t=4.385, p<0.001, respectively). BIG1 (t=2.455, p<0.05) and SML1 (t=2.916, p<0.05) males had higher corticosterone levels than C1 males. Female BIG1, BIG2, SML1 and SML2 also had higher corticosterone levels than female C1 (t=3.90, p<0.001; t=3.73, p<0.001; t=2.91, p<0.05 and t=4.91, p<0.001, respectively) and C2 (t=4.95, p<0.001; t=4.85, p<0.001; t=3.99, p<0.001 and t=5.30, p<0.001, respectively).

4. Discussion

In this study we report for the first time that females of both the BIG strains displayed face validity as a non-induced compulsive-like model by using more cotton for nest building and burying more marbles compared to the females of the SMALL strains. For nest-building behavior, the female Control strains were intermediate but closer to the SMALL strains, while for marble burying one of the Control strains was similar to the BIG strains and the other Control strain was similar to the SMALL strains. BIG1 and BIG2 females had less compulsive-like nesting (48% and 28%, respectively), when compared to BIG1 and BIG2 males establishing a sex difference, which was specific to the compulsive-like mice because the SMALL and Control strains did not show differences between males and females. We hypothesize that this difference was due to using proestrus females in this study with higher estrogen levels. For marble burying a similar sex difference was also observed for the BIG1 strain, but not the BIG2 strain although the trend was in the same direction.

Prior studies have shown that compulsive-like behaviors were enhanced in the absence of estrogen, such as estrogen-deficient male mice that showed development of compulsive-like wheel running and grooming behavior. These behaviors were considerably reduced with estrogen replacement [58]. Estrogen administration in ovariectomized rats with 8-OH-DPAT induced compulsive-like traits caused significant reduction in the compulsive-like behavior [33]. Females in proestrus have high physiological circulating estrogen levels [22, 99], but whether estrogen played a direct role in reducing compulsive-like behaviors in proestrus female compared to male compulsive-like mice remains to be elucidated.

Cohort based clinical studies have found that overall males have higher vulnerability to obsessions and compulsions when compared to females [14, 53]. Although obsessions are difficult to model in animals, the severity of compulsions can be a suitable measure of it [34]. Therefore, the higher levels of compulsive-like behaviors in BIG males compared to BIG females may further add face validity to the mouse model for understanding sex differences that attribute to the complexity of OCD.

Open field and elevated plus maze tests were used to assess anxiety-like behavior in males and females of BIG, SMALL and Control strains. No significant strain effects were found for the number of central entries in the open field, which indicates that this measure of anxiety-like behavior did not correlate with the level of compulsive-like behavior. However, significant strain differences were observed in elevated plus maze tests for both males and females. Generally the BIG strains showed less anxiety-like behavior in the elevated plus maze test when compared to the SMALL strains, which is consistent with our previous findings [50], and the females showed this general pattern most clearly. The responses of the six strains in the open field and elevated plus maze varied, which may be due to these tests measuring different aspects of emotionality associated with anxiety [6, 93].

Although BIG mice overall were less anxious than the SMALL mice in the elevated plus maze, within each BIG strain the correlations were generally opposite. Elevated plus maze exploration had a negative correlation with levels of compulsive-like nesting (r = −0.708, p<0.005) and marble burying (r = −0.562, p<0.05) behavior in BIG2 females indicating that an increase in compulsive-like behavior was associated with more anxiety (less exploration of the open arms). A negative correlation was also found between marble burying (but not nesting) with open arm exploration in BIG1 males (r = −0.817, p<0.0001). This result could be due to a changing genetic correlation structure as selection progressed, similar to what we found in these mice for the relationship between food consumption and nest building previously [19]. Alternatively, genetic drift and founder effects [17, 19] might be responsible for the correlations observed in just the BIG2 females and the BIG1 males.

The association of OCD with general anxiety is deemed controversial [103]. Studies have shown that the ego-dystonic and intrusive nature of obsessions differ largely from general anxiety [64, 65]. There is also large heterogeneity and variability of anxiety in OCD, though general anxiety is a common comorbid psychiatric condition associated with the disorder [5, 84]. This makes it an ambiguous indicator for the disorder [85]. General anxiety and panic disorders have a late onset when compared to OCD, which has a very early onset [95]. In addition, neurocircuitry models proposed for anxiety and OCD differ. While anxiety and panic disorders have been linked to dysregulation of amygdala-ventromedial prefrontal cortex-hippocampus circuitry [98], OCD is thought to occur due to abnormal fronto-striatal circuitry [36, 49, 88, 94]. Consequently, findings from the current study will allow the BIG2 female and BIG1 male strains to be used for studying association between various forms of compulsivity and anxiety. The BIG1 females and BIG2 males on the other hand can provide understanding of various forms of compulsivity and how they vary based on sex and genetic background.

In the depression-like forced swim test, no significant relationship was found between the level of compulsive-like behavior and immobility times as the strain effect was not significant. However, significant sex differences were observed with the females generally showing less depression-like behavior than males. Rodent studies have shown sex and strain differences in depression-like behavior in the forced swim task [13, 102]. Proestrus female mice and rats with higher circulating estrogen have been found to be less immobile in forced swim task when compared to the male counterparts [43, 44, 47], which is similar to our findings. Depression is a common comorbidity associated with OCD [5, 87, 104], but has been found to vary considerably among studies [3]. Human studies have shown that anxiety and depressions are influenced by factors like race/ethnicity and gender differences [78]. Hence, an effect of genetic background coupled with sex differences could influence the comorbid depression disorders in OCD and present a complex set of interactions, also reflected in our six mouse strains with different compulsive-like phenotypes.

The novel object recognition test revealed no significant strain effects, although the Control strains tended to outperform the BIG and SMALL strains. Females significantly outperformed the males, mostly due to the Control strains and less so due to the SMALL strains. The novel object recognition task is useful for studying working memory [38, 46, 100]. Several clinical studies have reported deficits in working memory among OCD patients [75, 81, 110], while others have reported no significant difference [85]. Our findings in the mice may be due to the fact that differences in working memory capacity among OCD patients are linked to intrusive thoughts or obsessions [15], which simply cannot be assessed in animals. Alternatively, novel object recognition memory may not be equivalent to the working memory deficits measured in OCD patients.

The female Control mice had significantly lower plasma corticosterone levels than the BIG or SMALL strain females. This general pattern was also observed in the males, but to a lesser degree. Previous studies have shown that female rodents secrete higher levels of corticosterone than males [10, 21, 66], which the BIG1, BIG2, and SML2 mice tended to show as well, although the overall sex effect was not significant. Additionally, in proestrus, corticosterone levels in response to stress are higher [9, 21, 35, 112]. Coping with stress is promoted by the hypothalamic-pituitary-adrenal (HPA) axis [28, 106] and corticosterone is a primary end point of HPA axis activation in mice [54]. OCD is known to be stress responsive since symptoms not only increase during periods of stress but stressful events can also precede the onset of obsessive-compulsive symptoms [80]. Our results show that selection to behavioral extremes, both in the high and low direction of nesting, resulted in enhanced activation of the HPA axis due to stress imparted by the forced swim test. OCD patients have higher baseline urine [45] and blood [61] cortisol levels than healthy matched controls, which can be compared to the higher stress-induced corticosterone levels in the BIG strains compared to the Control strains.

Significant replicate effects nested within strain were found for nest building, marble burying, open field behavior, novel object recognition memory, and the forced swim test. These replicate effects were most likely due to differences in genetic background as a result of random genetic drift and founder effects within each strain [17, 19]. Of special interest are the replicate effects between the BIG strains as they represent the compulsive-like phenotype and may correspond to subtypes of compulsive-like phenotypes as seen in human OCD patients [39, 48, 68]. The male BIG1 strain nested more than the BIG2 strain, while the BIG2 strain showed more anxiety-like behavior in the elevated plus maze than the BIG1 strain. However, the females were not different from each other for either trait indicating a potential sex by genotype interaction. Further, sex differences in only the BIG1 but not the BIG2 strains in marble burying add heterogeneity based on specific compulsive-like traits that might significantly vary depending on the sex and genotype.

5. Conclusion

In the current study, we confirmed that females of the compulsive-like BIG strains also have face validity as a mouse model of OCD, enhancing our earlier findings in males [50]. Proestrus females also showed lower levels of compulsive-like behaviors than males, which suggest that physiological changes in females related to the estrous cycle might influence compulsive-like behavioral expression. A differential response was observed in anxiety-like behaviors with replicate effects in the open field and strain differences in the elevated plus maze. Sex differences were seen for depression-like behavior with an elevated HPA axis response in females compared to males. Overall, our mouse strains can be used to better understand how genetic and sex factors and behavioral correlates contribute to the compulsive-like phenotype. Replicate effects in compulsive-like expression also indicate symptom heterogeneity that is strongly associated with OCD making it a complex neuropsychiatric disorder. Future studies will aim at investigating sex and strain differences in first line therapy responses and the neurobiological mechanisms in the mouse model that could broaden the understanding of heterogeneity and drug unresponsiveness in certain OCD subtypes when compared to others.

Highlights.

• Big (BIG) nest-building mice are a spontaneous animal model of OCD.

• BIG male mice showed higher levels of compulsive-like behaviors than proestrus females.

• Replicate effects were exhibited in anxiety-like open field behavior.

• Compulsive-like strains showed less anxiety-like behavior than the non-compulsive-like strains on the elevated plus maze.

• Novel object recognition memory was not differentially affected by level of compulsivity.

• Depression-like behavior was greater among males than proestrus females.

• Interplay of sex and strain (genetic background) affected the compulsive-, anxiety- and depression-like behaviors.