Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 31.
Published in final edited form as: Brain Behav Immun. 2014 Oct 7;44:176–186. doi: 10.1016/j.bbi.2014.09.018

ADVANCING THE UNDERSTANDING OF BEHAVIORS ASSOCIATED WITH BACILLE CALMETTE GUÉRIN INFECTION USING MULTIVARIATE ANALYSIS

Sandra L Rodriguez-Zas 1,2,*, Scott E Nixon 3, Marcus A Lawson 4, Robert H Mccusker 1, Bruce R Southey 1, Jason C O’Connor 5, Robert Dantzer 6, Keith W Kelley 1
PMCID: PMC4275396  NIHMSID: NIHMS639775  PMID: 25300921

Abstract

Behavioral indicators in the murine Bacille Calmette Guérin (BCG) model of inflammation have been studied individually; however, the variability of the behaviors across BCG levels and the mouse-to-mouse variation within BCG-treatment group are only partially understood. The objectives of this study were: 1) to gain a comprehensive understanding of sickness and depression-like behaviors in a BCG model of inflammation using multivariate approaches, and 2) to explore behavioral differences between BCG-treatment groups and among mice within group. Adult mice were challenged with either 0mg (saline), 5mg or 10mg of BCG (BCG-treatment groups: BCG0, BCG5, or BCG10, respectively) at Day 0 of the experiment. Sickness indicators included body weight changes between Day 0 and Day 2 and between Day 2 and Day 5, and horizontal locomotor activity and vertical activity (rearing) measured at Day 6. Depression-like indicators included duration of immobility in the forced swim test and in the tail suspension test at Day 6 and sucrose consumption in the sucrose preference test at Day 7. The simultaneous consideration of complementary sickness and depression-like indicators enabled a more precise characterization of behavioral changes associated with BCG-treatment and of mouse-to-mouse variation, relative to the analysis of indicators individually. Univariate and multivariate analyses confirmed differences between BCG-treatment groups in weight change early on the trial. Significant differences between BCG-treatment groups in depression-like behaviors were still measurable after Day 5. The potential for multivariate models to account for the correlation between behavioral indicators and to augment the analytical precision relative to univariate models was demonstrated both for sickness and for depression-like indicators. Unsupervised learning approaches revealed the complementary information provided by the sickness and depression-like indicators considered. Supervised learning approaches using cross-validation confirmed subtle differences between BCG-treatment groups and among mice within group identified by the consideration of sickness and depression-like indicators. These findings support the recommendation for multivariate and multidimensional analyses of sickness and depression-like indicators to augment the systemic understanding of the behavioral changes associated with infection.

Keywords: sickness, depression-like indicator, cluster analysis, principal component analysis, discriminant analysis

Introduction

The relationship between inflammation and depression in humans and in animal models is well-established. Individuals receiving immunotherapies have a higher incidence of depressive symptoms (Capuron and Miller, 2011). Patients with major depressive disorders have higher levels of serum pro-inflammatory cytokines than healthy controls (Maes, 2011). Likewise, depressive phenotypes were observed in response to bacterial challenge (Brydon et al., 2008). These associations suggested that inflammation may result in depressive symptomatology mediated by neuroimmune mechanisms. Designed experiments using animal models are offering insights into the relationship between infection, inflammation, and depression-like indicators. Mice injected live attenuated Bacille Calmette-Guérin (BCG) displayed high circulatory proinflammatory cytokines and indoleamine 2,3-dioxygenase activity. These mice exhibited sickness behaviors encompassing reduction in body weight and locomotor activity from Day 5 to Day 7. Likewise, challenged mice demonstrated depressive-like behaviors including lower mobility in the tail suspension test and in the Porsolt forced swim test, and lower sucrose intake in the sucrose preference test from Day 7 to Day 30 after treatment (Moreau et al., 2008; O’Connor et al., 2009). In addition, substantial mouse-to-mouse variation in response to BCG treatment was reported, including up to 30% of treated mice failing to exhibit adverse mobility effects (Platt et al., 2013).

Reductionist approaches based on the analysis of individual components have dominated the study of complex behavioral responses to infection. However, these reductionist approaches could have hindered the identification and characterization of systemic responses across multiple and typically correlated behaviors.

Six studies reported associations between BCG-treatment and sickness and depression-like behaviors in mice (Moreau et al., 2008; O’Connor et al., 2009; Kelley et al., 2013; Painsipp et al., 2013; Platt et al., 2013; Vijaya Kumar et al., 2014). In these studies, behavioral indicators were analyzed separately. This univariate approach could lead to substantially lower precision to detect differences between treatment groups, especially when the number of mice per group is low. Consideration of the covariation between indicators through multivariate approaches could enhance the precision to characterize smaller differences between treatment groups and the statistical significance of the BCG-treatment effect. Furthermore, the consideration of multiple indicators simultaneously could enhance the characterization of mouse-to-mouse variation and strengthen the identification of behavioral outliers. However, higher precision could come at the expense of higher number of parameters that need to be specified or estimated. Whether this trade-off results in a net benefit of better understanding the relationship between infection and sickness and depression indicators needs to be investigated.

The objectives of this study were: 1) to gain a comprehensive characterization of behavioral indicators in the BCG model of inflammation using multivariate approaches, and 2) to uncover behavioral differences associated with BCG-treatment level. Supporting activities include the consideration of complementary multidimensional approaches and study of mouse-to-mouse variation.

Materials and Methods

Samples

Adult (12 to 14 weeks old) male C57BL/6J mice from the Charles River Laboratory were studied. Mice were housed in individual cages under a normal 12:12 h light/dark cycle in a temperature- (23°C) and humidity- (45%) controlled room. Mice were offered water and food ad libitum (Teklad 8640 chow, Harlan Laboratories, Indianapolis, IN, USA) and handled daily for one week prior to the trial to ensure adaptation. Within the light cycle (lights on 10:00 PM–10:00 AM), behavioral tests began during the start of the dark phase under red lighting (O’Connor et al., 2009).

Three doses of the BCG strain of Mycobacterium bovis were studied. Live attenuated mycobacteria TICE BCG (50 mg wet weight of lyophilized culture containing 1×108 colony forming units or CFU/vial) was used (Organon USA Inc., USA).Vial reconstitution prior to inoculation used preservative-free saline and followed the provider’s instructions. Individual mice were challenged once with either 10 mg/mouse (BCG10 group, n=5), 5 mg/mouse (BCG5 group, n=6) or sterile saline solution (BCG0 group, n=7) at Day 0 of the experiment. Treatments were standardized to 0.3 ml/mouse and administered via intraperitoneal injection. Each mice was measured for the same set of sickness and depression-like indicators and thus, the measurements from 18 mice (5 mice BCG10 + 6 mice BCG5 + 7 mice BCG0) were analyzed. Experiments and measurements were implemented in accordance with the Animal Care and Use Program established by the University of Illinois at Urbana-Champaign Institutional Animal Care and Use Committee.

Sickness and depression-like indicators

The behavioral measurements are described in the sequence they were obtained. The measurements started early on the dark phase of the light cycle and behavioral experiments were performed during the first 7 h of the dark phase of the light cycle and followed established protocols (O’Connor et al., 2009; Lawson et al., 2013). Body weight changes following the BCG challenge was one indicator of sickness. Changes in body weight between Day 0 and Day 5 reflected the impact of infection on sickness through anorexia and modifications to metabolic homeostasis. Recovery from sickness was inferred from the subsequent increase in weight and similarity in locomotor activity and rearing between BCG treated and untreated mice at Day 6. Body weight was the first measurement and was recorded early in the dark phase of the light cycle. Daily measurements started on Day -1 to record the baseline weight. Locomotor activity measurements reflected the complementary impact of infection on sickness through fatigue and apathy for exploration. Horizontal movements (termed locomotor activity) and vertical locomotor activity (termed rearing) were measured at Day 6 in a novel cage using an established protocol for the open field method (O’Connor et al., 2009). Briefly, individual mice were placed in a standard acrylic cage including opaque walls and an insert dividing the floor into quadrants. The movements of the mice during five minutes were video recorded and counted by a trained observer that was blind to the treatment assignments. Locomotor activity was measured as the number of times the mouse crossed one of the grid lines with all four paws and rearing was measured as the number of times the mice stood on their hind legs either along a wall or independently (Brown et al., 1999).

Complementary depression-like indicators that reflect despair- and reward-based behaviors were measured. The duration of immobility in the tail suspension test and in the forced swim test at Day 6 were used as indicators of despair-based behaviors (Castagné et al., 2011). Sucrose intake in the sucrose preference test at Day 7 was used as indicator of anhedonia and a reward-based behavior (Strekalova et al., 2012).

The forced swim test followed the locomotor activity test (O’Connor et al., 2009). In the forced swim test, mice were placed in a cylinder containing 15-cm-high water that is approximately 23°C. After placing the mice in the water, the activity was recorded for 6 min. The duration of immobility was measured during the final 5 min by a trained observer (O’Connor et al., 2009). Applying published protocols, the tail suspension test followed the forced swim test (O’Connor et al., 2009). Mice were suspended by their tails from a hanger linked to a load cell for 10 min. The force transducer detected movements and the seconds spent motionless or immobile per minute were automatically recorded using the Mouse Tail Suspension package (MED-TSS-MS; Med Associates Inc., St. Albans, VT, USA). The average time that a mouse remained motionless per minute between 3 and 8 min post suspension was used as an indicator of immobility to remove extreme behaviors at the start and end of the trial. The spacing between the sickness locomotor and depression despair measurements within a mouse was on average 2.4 hours. The sucrose preference test was administered following the methodology described by Lawson et al. (2013). Mice had access to water and a 1% (wt/vol) sucrose solution, each available from a separate bottle. On Day -2 prior to treatment, mice were trained by simultaneous presentation with a bottle of water and a bottle of 1% (wt/vol) sucrose solution. Consumption of water and sucrose was measured by weighing the bottles after a 24-hour period. Sucrose preference was measured as the sucrose consumed relative to the total water and sucrose consumed, expressed as percentage.

Statistical analysis

A comprehensive analysis of the changes in behavior associated with BCG challenge was undertaken using complementary univariate and multivariate approaches including linear models, unsupervised and supervised learning, and multidimensional reduction and scaling techniques. These techniques were applied to accomplish two goals: the identification of groups of mice and the identification of groups of sickness and depression-like indicators. Widely used methods readily available in commonly used statistical software and packages are presented and their applicability to study sickness and depression-like indicators demonstrated. Unsupervised learning approaches that do not use information on the BCG treatment received by the mice can revealed the distinct and complementary information provided by the sickness and depression-like indicators considered. Supervised learning approaches that consider the sickness, depression-like and treatment information can confirm that the identification of subtle differences in behaviors between BCG-treatment groups and between mice within group.

The workflow to analyze multiple behavioral indicators and gain a comprehensive understanding of the impact of BCG treatment included four stages: 1) characterization of sickness and depression-like indicators using univariate and multivariate linear model analyses, 2) discovery of clusters of mice and clusters of indicators using hierarchical cluster analysis; 3) uncover relationships between mice within and between BCG-treatment groups and between sickness and depression-like indicators using multidimensional reduction and scaling; and 4) development of markers to accurately classify mice into BCG-treatment groups using discriminant analysis and k-nearest neighbor and confirmation of the classification using leave-one-out cross-validation. The algorithms used in this study are widely used and available in multiple statistical packages and languages including SAS (SAS Institute Inc. 2013) and R (R Core Team 2012). The corresponding procedures available in the previous statistical packages are also noted.

Univariate linear models

Linear models enable the description of the sickness or depression-like indicators or dependent variables in relationship to a number of independent variables. Estimates of the relationship between dependent and independent variables enabled assessment and testing of the strength of these relationships. Body weight was measured across multiple days within each mouse and thus a repeated measurement linear mixed effects model was used to describe the change in body weight across days and BCG-treatment groups. The model included the fixed effects of BCG-treatment level (BCG0, BCG5, and BCG10), day (Day 0 to 5), the interaction among BCG treatment group and day and body weight at Day -1. Preliminary tests indicated that the repeated structure of the measurements was adequately modeled by an autoregressive order 1 structure including heterogeneity of variances across days and mouse was the experimental unit.

Univariate linear models were used to describe the change in weight between Day 0 and Day 2, the change in weight between Day 2 and Day 5, locomotor activity, rearing, immobility in the forced swim and tail suspension tests, and sucrose preference. These models included the classification fixed effect of BCG-treatment level and the covariate body weight at Day -1. Additional terms were included in the models of specific indicators. Models describing sickness indicators included as covariates depression-like indicators meanwhile models describing depression-like indicators included as covariates sickness indicators. This strategy enabled the study of the effect of BCG challenge on sickness or depression-like indicators adjusted for depression-like or sickness, respectively. Covariates were nested within BCG-treatment group to account for the different trends of the covariates within group. Evaluation of the differences between observed and predicted values enabled the identification of possible outliers and assessment of departures from the normality assumption. For the sample size available, the statistical significance of parametric tests was confirmed using a non-parametric resampling approach including 10,000 bootstrap samples. Resampling followed PROC MULTEST and the merBoot method in SAS and R, respectively.

Multivariate linear models

While univariate models describe one indicator at a time, multivariate models consider multiple response indicator variables. Multivariate models are advantageous when the response variables are correlated through the signal or noise components of the model. There is a compromise between the gains in precision to detect the relationship between the indicators and explanatory variables and the additional parameters in the multivariate relative to the univariate models (Stearns et al., 2005; Serão et al., 2013). In a multivariate analysis, the test statistics available to assess the association between BCG-treatment group and behavioral indicators are equivalent when comparing two groups. Thus, results from one test, the Roy’s greatest characteristic root are presented. The multivariate models included the same cofactors and covariates used in the univariate models. Within the univariate and multivariate analyses, estimable functions were used to compute the expected behavior indicator value within BCG-treatment group at the average value of all other covariates. Univariate and multivariate models specification followed PROC MIXED and the lme4 package in SAS and R, respectively.

Measurements were inspected for the presence of outliers that severely deviated from biological or statistical expectations and could impact the estimates and test statistics. Within linear models, outliers were identified using the standardized residuals. Within unsupervised learning approaches, outliers were identified based on the Euclidean distance between pairs of mice based on all behavioral indicators. Distances were computed using PROC DISTANCE or the dist function in SAS and R, respectively.

Cluster analysis

Using the unsupervised learning method of cluster analysis mice were grouped into clusters of similar behavioral profile. Likewise, cluster analysis enabled the grouping of sickness and depression-like indicators into clusters of similar profile and uncovered relationship between these indicators. Mice were clustered based on weight change between Day 0 and Day 2, weight change between Day 2 and Day 5, locomotor activity, rearing, tail suspension immobility, forced swim immobility, and sucrose preference. Also, the behavior indicators were clustered based on information from all 18 mice across the three BCG-treatment groups. Through hierarchical agglomerative clustering, mice (or indicators) were grouped in a sequential manner from lower to higher Euclidean distance while minimizing the within cluster variation until all items were part of one cluster (Kaufman and Rousseeuw, 2005). A dendrogram or tree diagram was used to represent the distance between items (mice or indicators) or between clusters. The distance between items was represented by the branch length. The number of clusters supported by the data was inferred from the changes in the within and across cluster variation along the clustering process. Routines including PROC CLUSTER and the hclust function are the SAS and R alternatives, respectively.

Multi-dimension reduction and scaling approaches

Insights from cluster analysis were complemented with multidimensional approaches that use the information from multiple orthogonal variables to further understand the relationship between the behavior indicators. The dimensions reduced or scaled were the weight change between Day 0 and Day 2, weight change between Day 2 and Day 5, locomotor activity, rearing, tail suspension immobility, forced swim immobility and sucrose preference.

Principal component analysis (PCA) and multidimensional scaling (MDS) were used to identify a reduced number of indices (functions of the behavioral indicators measured) that portrayed the main relationships among mice within and between BGC-treatment groups (Zuur et al., 2007). Each principal component (in PCA) or scale (in MDS) accounted for a non-overlapping percentage of the variation of the original behavior indicators in an inverse relationship between component number and percentage of the variation explained. In this study, two paths were explored. First, the correlation between the behavioral indicators was used to infer the coefficients (or loadings) of these indicators and the relationship between mice. Second, the correlation between mice was used to infer the relationship between the behavioral indicators. The Pearson’s correlation coefficient between the indicators was favored over the covariances to level the impact of indicators despite differences in magnitude.

For dimension reduction purposes, the components or scales considered were limited to those that explained most and together accounted for at least 70% of the variance of the original measurements. The relationship between sickness and depression-like indicators and the relationship between mice within and across BCG-treatment groups was investigated through the evaluation of the coefficients of the variables in the first principal components together with the visualization of the relative location of the mice from different BCG-treatment groups along pairs of major principal components.

An analysis comparable to PCA was implemented using multidimensional scaling. This approach relied on the distances between items and double-centering of the distance matrix instead of correlations used in PCA. Thus, the consistency between MDS and PCA outputs depended on the properties and structure of the original measurements. Implementation of PCA includes PROC PRINCOMP and the princomp function in SAS and R, respectively. Implementation of MDS includes PROC MDS and the cmdscale function in SAS and R, respectively.

Supervised learning approaches

Supervised learning approaches that account for the known BCG-treatment assignment were used to develop decision rules that assigned mice to classes (i.e. BCG-treatment groups) with maximum possible accuracy (Zuur et al., 2007). Supervised prediction of mice classification into BCG-treatment groups was based on weight change between Day 0 and Day 2, weight change between Day 2 and Day 5, locomotor activity, rearing, tail suspension immobility, forced swim immobility and sucrose preference. Consideration of the coefficients of the behavioral indicators in the classification functions offered insights into the relationship between indicators. Two complementary supervised learning methods, linear discriminant analysis (LDA) and k-nearest neighbor (KNN), were evaluated. In LDA, the resulting indices of the behavioral indicators offered the maximum distance between the observed classes and the minimum variation within class. Mice were assigned to the class that was most proximal to the LDA index value. In the KNN approach, mice were assigned to the class of all or most of the closest neighboring mice based on the Euclidean distance. The LDA and KNN approaches are implemented in the PROC CLUSTER and lda and knn functions in SAS and R, respectively.

In summary, the linear univariate model analysis was evaluated because this approach is frequently used in the identification of associations between the variation in behavioral indicators and BCG treatment. The multivariate model is a statistically well-understood extension of the univariate approach with comparable type of outputs. Meanwhile linear models require the identification of a response and explanatory variables, unsupervised learning does not require treatment group information. The results from PCA and MDS supplement those from cluster analysis. While cluster analysis identifies groups of variables (mice or behavior indicators) alike (based on indicators or mice, respectively), PCA and MDS aid in the identification of fewer combinations of the original variables (mice or behavior indicators) that represent information comparable to the original variables. Lastly, the supervised learning approaches LDA and KNN utilize the treatment information from a number of observations to assign a treatment group to the remaining observations. The cross-validation implementation permitted the classification of one mouse using a classifier function developed on the remaining mice.

RESULTS AND DISCUSSION

A number of approaches were used to further understand the impact of BCG-challenge on behavior indicators in a mouse model of inflammation-induced depression. This study also investigated the changes in sickness and depression-like indicators associated with BCG-treatment levels and mouse-to-mouse variation. Both, the relationships among mice within a BCG-treatment level and among behavior indicators were investigated. No mouse was removed from the analysis because 1) no observation exhibited an extreme standardized residual in the linear model analyses and, 2) no extreme Euclidean distances between mice were detected as part of the unsupervised learning analyses.

For baseline purposes, results from the analysis of individual behavioral indicators using univariate linear model analyses are presented first. The univariate results served as point of reference for comparison against results from previous studies and against results from multivariate linear model analysis and supervised and unsupervised learning approaches. Additional multivariate insights on the relationship between mice and between behavior indicators were gained from cluster, multidimensional reduction and scaling and discriminant analyses.

Univariate and multivariate linear model analyses of sickness and depression-like indicators

The testing of differences in behavioral indicators between BCG-treatment levels using standard univariate models enabled benchmarking the studied mice population and BCG-challenge against published studies. Results from the univariate analyses validated the phenotypic trends reported in related studies (Moreau et al., 2008; O’Connor et al., 2009). This validation also confirms that the sample studied is consistent with population expectations.

Univariate linear mixed model analysis of body weight from Day 0 to Day 5 demonstrated that the significant differences in body weight among the three BCG-treatment groups by Day 2 were no longer significant by Day 5 (Figure 1). The drop in weight between Day 0 and Day 2 was −0.36 (±0.27), 1.62 (±0.30), and 2.26 (±0.33) for groups BCG0, BCG5 and BCG10, respectively. Principal component analysis of daily body weights between Day 0 and Day 5 uncovered that two major trends (pre and post Day 2) explained 99% of the variation across the six days. Consideration of the coefficients in the PCA eigenvectors indicated two body weight patterns (before and after Day 2) that were consistent with the linear model findings. These results are in agreement with previous reports that BCG-challenged mice lose weight until Day 1 or Day 2 and subsequently gain weight (Moreau et al., 2008; O’Connor et al., 2009). Based on these findings, two weight indicators of sickness were used: weight change between Day 0 and Day 2 and weight change between Day 2 and Day 5. These two measurements were computed as the difference in weight between the last and the first time point. These two measurements captured the two main weight change trends.

Figure 1.

Figure 1

Open in a new tab

Body weight across days post injection (at Day 0) and across BCG-treatment level [BCG0 (n=7), BCG5 (n=6), and BCG10 (n=5)] estimated from a repeated-measurements longitudinal model. Estimates were adjusted for weight prior to the start of the experiment and for heterogeneity of variance. Whiskers denote 95% confidence limits of the estimates.

Results from the univariate linear models indicated a significant association between BCGtreatment and both change in weight between Day 0 and Day 2 (P-value < 0.0027) and change in weight between Day 2 and Day 5 (P-value < 0.0046). The models for these indicators accounted for more than 80% of the variation of weight (R2 > 80%). Among the BCG-treated mice, the BCG10 group had the highest (P-value < 0.0024) weight loss between Day 0 and Day 2 relative to BCG0 followed by BCG5 (P-value < 0.003) meanwhile the difference in weight change between the BCG10 and BCG5 groups was non-significant. Among the BCG-treated mice, the BCG5 group had the highest (P-value < 0.0014) weight gain between Day 2 and Day 5 relative to BCG0 followed by BCG10 (P-value < 0.032) meanwhile the difference in weight change between the BCG10 and BCG5 groups was non-significant.

The multivariate analysis of both weight change indicators improved the precision, identifying an association between BCG treatment and weight more significant (Roy’s greatest Root P-value < 0.0010) than the univariate analyses (P-value < 0.0027 and P-value < 0.0046). These results are in agreement with previous studies using a number of mice strains and genotypes where BCG-challenged mice exhibited a drop in weight during the first 2 days post-challenge followed by a weight gain (Moreau et al., 2008; O’Connor et al., 2009; Platt et al., 2013; Painsipp et al., 2013; Vijaya Kumar et al., 2014). The speed of recovery of body weight varies with study and strain and meanwhile in some studies body weight does not differ among BCG-treated and BCG0 mice by Day 6 (Pratt et al., 2013; Painsipp et al., 2013), in other studies weight recovery is detected after Day 7 (Moreau et al., 2008; Kelley et al., 2013; Vijaya Kumar et al., 2014).

Results from univariate linear models indicated a non-significant (P-value > 0.1) association between BCG-treatment and locomotor activity and rearing. Likewise, the bivariate analysis of both activity indicators of sickness detected a non-significant BCG treatment effect (Roy’s greatest Root P-value > 0.1). These results are consistent with previous reports that BCGchallenged mice no longer exhibited significant sickness symptoms by Day 6 (Moreau et al., 2008; Platt et al, 2013). Deficits in locomotor activity in BCG-induced mice were nearly resolved by Day 1 and were non-significant by Day 7 (Platt et al., 2013; Kelley et al., 2013). One study reported borderline non-significant differences in locomotor activity by Day 7 in C57BL/6N mice (Painsipp et al., 2013) meanwhile a different study using C57BL/6J mice reported non-significant differences in rearing yet significant differences in horizontal locomotor activity after Day 7 (O’Connor et al., 2009). Another study using BALB/c mice found nonsignificant differences in total distance traveled by Day 14 post-challenge, although differences were still significant by Day 7 (Vijaya Kumar et al., 2014).

The results from the univariate linear model analysis indicated a significant (P-value < 0.0336; R2 = 71%) BCG-treatment effect on tail suspension immobility. In particular, a significant (Pvalue < 0.0363) difference in mobility between BCG-treated and non-treated groups was detected. These results are consistent with previous reports that immobility measured by tail suspension test persisted beyond sickness behaviors after Day 7 (Moreau et al., 2008; O’Connor et al., 2009; Platt et al., 2013; Kelley et al., 2013; Vijaya Kumar et al., 2014). A borderline significant (P-value > 0.09; R2 = 59%) difference between BCG-treated and non-treated mice groups was detected for forced swim immobility. Mice in the BCG0 group remained immobile less time than BCG-treated mice and the immobility of BCG5 mice was closer to BCG10 than to BCG0 mice. The trends for sucrose preference followed a similar pattern albeit non-significant (P-value > 0.1). Mice in the BCG0 group exhibited higher sucrose consumption than BCG-treated mice and the sucrose consumption by the BCG5 mice was closer to BCG10 than to BCG0 mice. These findings are consistent with a previous report of non-significant differences in forced swim and sucrose preference indicators between BCG-treated and saline groups (Moreau et al., 2008). Similar to weight change, the application of multivariate analyses to the three depression-like indicators demonstrated the potential of this approach for to account for the correlation between indicators and to augment the analytical precision. A significant effect of BCG-treatment group on all three depression-like indicators and a significant difference between BCG-treated and non-treated groups was detected (Roy’s greatest Root P-value < 0.036). This association was identified despite the higher number of estimated parameters in the multivariate analysis compared to the univariate analyses and despite that the univariate analysis detected a non-significant association. Significant differences in the forced swim and tail suspension immobility between BCG-treated and saline-treated mice beyond Day 7 have been reported (O’Connor et al., 2009).

The visualization of the distribution of mice within and across BCG-treatment groups resulting from the cluster, principal component, and discriminant analyses revealed distinct behavioral patterns between mice in the BCG10 and BCG5 groups and also mouse-to-mouse variation within group. The multidimensional approaches demonstrated the distinct and complementary nature of sickness and depression-like indicators. These analyses also confirmed the behavioral differences between BCG-treated and non- treated mice. Multivariate unsupervised and supervised methods were used to identify both, groups of mice with similar behaviors and groups of behavioral indicators that exhibited similar profiles across mice.

Unsupervised learning analysis

Hierarchical cluster analysis was explored because this approach does not require the assumption of specific parameters describing the relationship between the variables considered. The dendrogram resulting from the hierarchical cluster analysis of mice is presented in Figure 2. The shorter the branch length of a dendrogram, the shorter the distance (the greater the similarity) between mice across the seven behavior indicators considered. The branch length was quantified using the semi-partial R2 that measures the increase in variability within cluster (relative to between clusters) resulting from the grouping of mice, partial on the number of clusters in each dendrogram level. The longest branches connected the three BCG treatment groups. Furthermore, mice from BCG0 group were more distant from the other groups relative to the distance between BCG5 and BCG10 mice. All except two mice were proximal to mice within the same BCG treatment group. The exceptions include one BCG5 mouse that was closer to a BCG10 mouse and one BCG10 mouse (mouse number 22) that was closer to a BCG0 mouse than to mice from their corresponding treatment groups. Results from complementary MDS analysis of the BCG10 mouse number 22 are presented in the MDS section.

Figure 2.

Figure 2

Open in a new tab

Dendrogram displaying the distance (branch length measured in semi-partial R2 units) between mice across BCG treatment groups. Mouse identification includes the leftmost vertical axis label denoting the BCG-treatment group [10 = BCG10 (n=5), 5 = BCG5 (n=6), or 0 = BCG0, (n=7)] followed by “_” and the mouse unique identifier number.

A previous study reported substantial mouse-to-mouse variation in the depression-like indicator immobility among CD-1 mice treated with BCG (Platt et al., 2013). In that study, up to 30% of BCG-treated mice did not exhibit increased immobility in the tail suspension test at Day 7 post treatment and these mice were categorized as “resilient” to BCG induced behavioral changes. The majority of BCG-treated mice exhibited increased immobility at Day 6 post treatment and were categorized as “susceptible”.

Further understanding of the relationship between behavioral indicators was gained from complementary disjoint cluster analysis using a divisive process. The dendrogram in Figure 3 depicts the relationship between indicators. The branch length or indicator of distance represents the proportion of the variance explained by the clustered indicators. As demonstrated in the dendrogram, the more distant the indicators being clustered (e.g. clustering of depression-like and sickness indicators relative to the clustering of two sickness indicators), the lower the proportion of the variance explained by the clusters. The disjoint procedure clearly demonstrates the complementary information offered by the sickness and depression-like indicators. The weight-change sickness indicators were clustered together and in proximity to the other sickness indicators, locomotor activity and rearing. The depression-like indicators were distant from all sickness indicators, and among these, the immobility indicators were more proximal to each other than to sucrose preference.

Figure 3.

Figure 3

Open in a new tab

Dendrogram displaying the distance (branch length measured in proportion of variance explained) between sickness indicators including change in weight between Day 0 and 2 (W20), change in weight between Day 2 and 5 (W52), horizontal locomotor activity (LMA) and rearing (REA), and depression-like indicators including immobility in the tail suspension test (TST), immobility in the forced swim test (FST) and sucrose preference test (SPT) based on n=18 mice.

Principal component analysis and multidimensional scaling

The dendrogram from hierarchical cluster analysis constituted the first step towards understanding the relationship between mice, treatment groups and behavioral indicators. However, the collapse of the distance information into one number (the branch length connecting the item or cluster to other clusters) may limit the understanding of the contributions of individual mouse or indicators to the relative distance between items and clusters. For example, the position of a mouse in the dendrogram may be the result of consistent patterns across all behavioral indicators or may be the result of an average across distinct patterns. Dimensional reduction and scaling approaches were considered to expand the understanding of the role of sickness and depression-like indicators on BCG-treatment grouping and of the role of mice from different BCG-treatment groups in the grouping of behavioral indicators.

The interpretation of the multivariate information from all seven sickness and depression-like indicators across mice and BCG-treatment groups was enhanced by the three main outcomes from PCA: a) the number of principal components that account for the majority of the variation of the original measurements; b) the coefficients of the variables in the major principal components; and c) visualization of the distribution of the items along the major principal components.

The plot of the first three principal components depicts the clear separation between mice in the BCG0 group, denoted by circles, and the other two BCG-treated groups (Figure 4). The first three principal components of the PCA used to identify the distribution of mice across the most informative and orthogonal dimensions, explained 70% of the variation of the seven original behavioral indicators. Meanwhile principal component 1 enabled the separation between BCG0 and BCG-treated mice, principal components 2 and 3 enabled the separation between BCG10 and 5 groups. As expected, the weaker differences in behavioral indicators between the two BCG-treated groups required additional principal components to distinguish the groups.

Figure 4.

Figure 4

Open in a new tab

Distribution of the mice in the BCG10 (red squares, n=5), BCG5 (orange diamonds, n=6) and BCG0 (green circles, n=7) across the first three principal components (Prin1, Prin2, Prin3).

The coefficients of the original behavioral indicators in the first three principal components confirmed the distinct patterns profiled by the sickness indicators. Figure 5 depicts the coefficients of all seven indicators in each of the three first principal components. Similar coefficient values within a principal component indicate that the behavioral indicators exhibited a similar covariation. The sickness indicators (change in weight between Day 0 and 2, change in weight between Day 2 and 5, locomotor activity and rearing) received the most extreme coefficients in principal component 1. The nearly opposite coefficients of both weight changes correspond to the opposite patterns of weight change prior and post Day 2 in BCG-treated mice. On the other hand, the similar coefficients received by the horizontal locomotor activity and rearing are consistent with the similar impact of BCG-treatment on both activity indicators at Day 6. The coefficients in principal components 2 and 3 distinguished sucrose preference from the other two depression-like indicators of immobility.

Figure 5.

Figure 5

Open in a new tab

Coefficients (or loadings) received by each sickness and depression-like indicator (based on n=18 mice) in the first three principal components (PC1, PC2, and PC3). Small circle: weight change between Day 0 and 2; large circle: weight change between Day 2 and 5; small square: horizontal locomotor activity; large square: rearing; small diamond: tail suspension immobility; medium diamond: forced swim immobility; and large diamond: sucrose preference.

The results from PCA supplement those from cluster analysis because meanwhile cluster analysis identifies groups of variables (mice or behavioral indicators) alike (based on indicators or mice, respectively), PCA is a process for identifying combination of the original variables (mice or behavioral indicators) that represent information comparable to the original variables. The outcome from cluster analysis is the grouping of the original variables based on a criterion (e.g. variation between versus within clusters) meanwhile the outcomes from PCA are linear indices of the original variables. The coefficients of the variables in the indices offer insights into the relationship between the original variables and this information is expected to be consistent or complementary to the relationships identified in the cluster analysis.

The PCA coefficients received by the behavioral indicators depicted in Figure 5 are consistent with the clustering of indicators presented in Figure 3. The pair of indicators locomotor activity and rearing and the pair of indicators tail suspension test and sucrose preference test appear closer to each other. The three dimensions of the PCA reported offer additional information to the one dimension of the lengths of the cluster tree branches. For example, meanwhile the weight change between Day 0 and Day 2 and the weight change between Day 2 and Day 5 received coefficients of similar magnitude for principal components 1 and 3, the magnitudes differ for principal component 2. Another complementary insight from the consideration of three principal components relative to cluster analysis is the characterization of the relationship between the three depression-like indicators. Sucrose preference received coefficients of similar magnitude to tail suspension and forced swim immobility for principal components 1 and 2 and different for principal component 3. This evaluation of the changes in the relationship between the coefficients across principal components further confirms the supplementary information provided by the three dimensions considered.

Multidimensional scaling

Multidimensional scaling is a class of method to map items and under specific circumstances, produces the same output as PCA. The complementary and confirmatory insights offered by MDS were evaluated. Results from MDS of the relationship between indicators confirmed the closer relationship within sickness and within depression-like indicators (Figure 6 left). Dimension 2 differentiated between sickness indicators (receiving positive coefficients) and depression-like indicators (receiving negative coefficients). Dimension 1 differentiated among the sickness indicators weight change (receiving positive coefficients) and activity (receiving negative coefficients).

Figure 6.

Figure 6

Open in a new tab

Left. Distribution of the behavioral indicators across multidimensional scaling dimensions 1 and 2. Right. Distribution of mice denoted by the BCG-treatment group [_0 = BCG0 (n=7), _5 = BCG5 (n=6), _10 = BCG10 (n=5)] followed by “_” and by the mouse identifier number from multidimensional scaling dimensions 1 and 3.

The overall consistency of results between the PCA and MDS analyses speaks to the strength of the relationships among the behavioral indicators measured in this study. The slight differences between the relative coefficients in the PCA and MDS implementations is related to the PCA identification of the linear combination of indicators that maximize the explained variance adjusted for all higher order combinations, meanwhile MDS preserved the distances between items while representing the items in a lower dimensional space.

The results from MDS confirmed the distribution of mice within and between BCG-treatment groups observed in the PCA (Figure 6 right). Mice from the BCG0 and BCG10 groups were located on either side of the two-dimensional plot, meanwhile mice in the BCG5 group were located in-between. Multidimensional scaling analysis offered insights into the relative behavior of BCG10 mouse number 22 that clustered closer to the BCG0 group. Figure 6 (right) demonstrates that this mouse was approximately half-way in between group BCG10 and BCG0. Closer inspection of the indicators revealed that despite exhibiting levels of horizontal locomotor activity, rearing, forced swim immobility, and sucrose preference consistent with other mice in the BCG10 group, this mouse maintained weight during the trial. The unique combination of levels displayed by this mouse suggests the need to consider multiple sickness and depression-like indicators simultaneously and the need to measure additional mice.

Linear Discriminant Analysis

Linear discriminant analysis enabled a perfect discrimination of the mice among the corresponding BCG-treatment groups without miss-assignments. Leave-one-out cross validation confirmed these BCG-treatment class assignments. The coefficients of the behavioral indicators in the indices that discriminate between BCG10, BCG5 and BCG0 offered insights into the impact of indicators in the discrimination between BCG-treated and BCG0 but also within BCG-treated groups (Table 1). A linear trend was observed between the coefficient of the indicator and the BCG-treatment level in all except two behavior indicators. The linear trend consists on an increase (or decrease) in the coefficient with BCG-treatment level. This trend slightly departed for the forced swim immobility; however, the difference in coefficients between BCG-treatment levels was only 10%. For the indicators change in weight between Day 2 and 5 and locomotor activity, the BCG5 group exhibited a substantially more extreme coefficient than the other two groups. These results suggest that the differences between both BCG-treated groups were driven by difference in sickness indicators, and in particular the capability to recover lost weight and to display horizontal locomotor activity. The difference between BCG-treatment groups in weight change was detected in the univariate analysis meanwhile the difference in horizontal locomotor activity was highlighted by linear discriminant analysis. These results confirm the additional insight offered by complementary approaches. Furthermore, mouse number 22 pertaining to group BCG10 was classified in the correct group. The tail suspension test measurement of mouse number 22 was the lowest of the group; however the value was not distant from the second lowest measurement.

Table 1.

Coefficients of sickness and depression-like indicators in the discriminant indices that assigned mice to the BCG0, BCG5 and BCG10 groups based on n=18 mice.

Behavior Indicator BCG0 BCG5 BCG10
Change in weight between Day 0 and 2 −2.22750 −3.22502 −3.73052
Change in weight between Day 2 and 5 1.72475 7.78472 3.25526
Horizontal locomotor activity 0.88061 1.10627 0.77754
Rearing 4.04187 3.82697 3.36034
Forced swim immobility 0.38761 0.28640 0.31652
Tail suspension immobility −0.14464 −0.05314 0.01539
Sucrose preference 1.89954 1.66150 1.63912
Open in a new tab

K-nearest neighbor method

Using the nearest neighbor mouse and the seven sickness and depression-like indicators, all mice were correctly assigned to the correct BCG-treatment group. Using the information on all seven sickness and depression-like indicators from the two most proximal neighbor mice, all BCG0 mice and all BCG10 mice were discriminated into the corresponding groups. Among the BCG5 group, four mice were assigned to the correct group and two mice were assigned to the BCG10 group. This result speaks to the mouse-to-mouse variability within BCG-treatment group and the between:within group variation. The two miss-classified BCG5 mice exhibited profiles similar to BCG10 mice. This result supports previous reports of varying levels of susceptibility of mice to BCG-challenge

Additional considerations

Laboratory effects on behavioral indicators including apparatus, test procedure, order of tests, and experimenter error have been widely recognized (Chesler et al., 2002a,b; Brown et al., 2007). Behaviors measured by a number of tests appear to be more sensitive to the previous testing experience than others (McIlwain et al., 2001). Alternative tests to measure sickness and depression-like indicators could offer complementary information on the association between BCG-treatment and behavior. Supporting this, multivariate approaches are well suited to handle additional behavioral indicators. However, care must be exercised to ensure that the order of a larger number of tests on the same subjects does not influence the measurements. Also, consideration of multiple mouse strains would enable the testing of synergistic or antagonistic relationships between strain and BCG-treatment on behavioral indicators in addition to the detection of treatment effects that are common to all strains. Recommendations for supervised and unsupervised analyses include the availability of at least five observations per variable (Stevens, 2009). In this study the multivariate linear model described two (weight changes or locomotor activity and rearing) or three variables (tail suspension test, forced-swimming test, and sucrose preference test). For these analyses, the 18 subjects available were adequate. The availability of more observations would augment the precision of the estimates and enable independent cross-validation.

CONCLUSIONS

Reductionistic and modular analysis of individual behavioral indicators may fail to capture trends that can become evident when multiple modules are considered simultaneously. This study compared the results from reductionistic and systemic multivariate or multidimensional approaches to understand the changes in behavioral indicators associated with infection status. Four sickness indicators and three depression-like indicators were measured in mice receiving either one of three BCG-treatment levels. Mice treated with BCG exhibited sickness as indicated by changes in body weight during the first days after the challenge. Although the difference in sickness indicators between BCG-treatment groups subsided by Day 5, differences in depression-like indicators were detected in subsequent days. The previous trends were weaker or less recognizable in the univariate reductionist analysis than in the multivariate systemic analysis. Our results showed that the classical univariate analysis of indicators individually may fail to capture borderline trends. This finding is important because detecting subtle differences between treatment groups or subjects within treatment group is becoming more critical with the recognition of the effectiveness of individualized therapies. Furthermore, the multivariate and multidimensional approaches offered information on the relationship between behavioral indicators and between mice within and across treatment groups, in addition to traditional test statistics.

Cluster, multidimensional reduction and scaling analyses further characterized the interplay between sickness and depression-like indicators. The distribution of mice across sickness and depression-like dimensions confirmed that the BCG5 treatment elicited weaker changes in sickness and depression-like indicators than the BCG10 treatment. Also, the distribution of mice within BCG-treatment group confirmed mouse-to-mouse variation on the susceptibility to challenge across the multiple behavior indicators. This result suggests that studies aimed at characterizing mouse-to-mouse variation may consider low BCG dose levels.

Subject-to-subject variation in behavioral response to infection and identification of differential susceptibility has been reported in humans (Walker et al., 2011). Beyond the polygenic nature of susceptibility to BCG challenge, results from the multivariate analysis suggested an epistatic mode of action of some genes across indicators. The impact of indoleamine 2,3-dioxygenase on the development of BCG-induced behavioral changes has been demonstrated (O’Connor et al., 2009). Resistance-associated macrophage protein 1, and interferon regulator factor 8 (IRF8) are among other genes associated with variation in the susceptibility to BCG infection (Fortin et al., 2007). Structural variation in these genes together with regulatory variation in the HPA axis activity are expected to influence inflammation-related sickness and depression-like behaviors. Future multivariate studies of sickness and depression-like behaviors in BCG-challenged mice will benefit from consideration of the concentrations of circulating pro-inflammatory cytokines and immune activation together with genomic sequence variation among mice.

The capability of multivariate models to enhance the precision to detect differences among BCG-treatment groups relative to univariate models was demonstrated both for weight changes indicators of sickness and for the three depression-like indicators. The unsupervised learning approaches demonstrated the distinct characterization provided by the sickness and depression-like indicators studied. This complementarity was confirmed by the supervised learning approaches. Therefore, multivariate analysis is recommended to establish models that enable understanding of complex interactions between various types of response to infection.

Highlights.

  • Sickness and depression indicators were measured in BCG-treated mice

  • Multivariate approaches can enhance the precision to detect BCG effects

  • Multidimensional analyses characterized the interplay between indicators

ACKNOWLEDGMENTS

The support of NIH grant numbers: R21 MH 096030, R01 MH 090127, R01 SUB UT 00000712, R01 MH083767, and USDA NIFA grant number 2012-38420-30209 are greatly appreciated.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  1. Brown R, Corey S, Moore A. Differences in measures of exploration and fear in MHC-congenic C57BL/6J and B6-H-2K mice. Behavior Genetics. 1999;29(4):263–271. [Google Scholar]
  2. Brown R. Behavioural phenotyping of transgenic mice. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale. 2007;61(4):328–344. doi: 10.1037/cjep2007033. [DOI] [PubMed] [Google Scholar]
  3. Brydon L, Harrison NA, Walker C, Steptoe A, Critchley HD. Peripheral inflammation is associated with altered substantia nigra activity and psychomotor slowing in humans. Biological Psychiatry; the Role of Sensitization in the Addiction Process. 2008;63(11):1022–1029. doi: 10.1016/j.biopsych.2007.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Castagné V, Moser P, Roux S, Porsolt RD. Rodent models of depression: forced swim and tail suspension behavioral despair tests in rats and mice. Current Protocols in Neuroscience. 2011;8(8.10A):1–14. doi: 10.1002/0471142301.ns0810as55. [DOI] [PubMed] [Google Scholar]
  5. Chesler EJ, Wilson SG, Lariviere WR, Rodriguez-Zas SL, Mogil JS. Influences of laboratory environment on behavior. Nature Neuroscience. 2002;5(11):1101–1102. doi: 10.1038/nn1102-1101. [DOI] [PubMed] [Google Scholar]
  6. Chesler EJ, Wilson SG, Lariviere WR, Rodriguez-Zas SL, Mogil JS. Identification and ranking of genetic and laboratory environment factors influencing a behavioral trait, thermal nociception, via computational analysis of a large data archive. Neuroscience and Biobehavioral Reviews. 2002;26(8):907–923. doi: 10.1016/s0149-7634(02)00103-3. [DOI] [PubMed] [Google Scholar]
  7. Fortin A, Abel L, Casanova JL, Gros P. Host genetics of mycobacterial diseases in mice and men: Forward genetic studies of BCG-osis and tuberculosis. Annual Review of Genomics and Human Genetics. 2007;8(1):163–192. doi: 10.1146/annurev.genom.8.080706.092315. [DOI] [PubMed] [Google Scholar]
  8. Johnson RA, Wichern DW. Applied multivariate statistical analysis. Prentice Hall: Upper Saddle Hall, New Jersey; 1998. [Google Scholar]
  9. Kelley KW, O'Connor JC, Lawson MA, Dantzer R, Rodriguez-Zas SL, McCusker RH. Aging leads to prolonged duration of inflammation-induced depression-like behavior caused by Bacillus Calmette-Guérin. Brain, Behavior, and Immunity. 2013;32:63–69. doi: 10.1016/j.bbi.2013.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Capuron L, Miller AH. Immune system to brain signaling: neuropsychopharmacological implications. Pharmacology & Therapeutics. 2011;130(2):226–238. doi: 10.1016/j.pharmthera.2011.01.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Lawson MA, Parrott JM, McCusker RH, Dantzer R, Kelley KW, O'Connor JC. Intracerebroventricular administration of lipopolysaccharide induces indoleamine-2,3-dioxygenase-dependent depression-like behaviors. Journal of Neuroinflammation. 2013;10:87. doi: 10.1186/1742-2094-10-87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Maes M. Depression is an inflammatory disease, but cell-mediated immune activation is the key component of depression. Progress in Neuro-Psychopharmacology and Biological Psychiatry; the Neuro-Inflammatory and Neuroprogressive Pathways in Depression. 2011;35(3):664–675. doi: 10.1016/j.pnpbp.2010.06.014. [DOI] [PubMed] [Google Scholar]
  13. McIlwain KL, Merriweather MY, Yuva-Paylor LA, Paylor R. The use of behavioral test batteries: effects of training history. Physiology and Behavior. 2001;73(5):705–717. doi: 10.1016/s0031-9384(01)00528-5. [DOI] [PubMed] [Google Scholar]
  14. Moreau M, André C, O'Connor JC, Dumich SA, Woods JA, Kelley KW, Dantzer R, Lestage J, Castanon N. Inoculation of Bacillus Calmette-Guerin to mice induces an acute episode of sickness behavior followed by chronic depressive-like behavior. Brain, Behavior, and Immunity. 2008;22(7):1087–1095. doi: 10.1016/j.bbi.2008.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. O'Connor JC, André C, Wang Y, Lawson MA, Szegedi SS, Lestage J, Castanon N, Kelley KW, Dantzer R. Interferon-γ and tumor necrosis factor-α mediate the upregulation of indoleamine 2,3-dioxygenase and the induction of depressive-like behavior in mice in response to Bacillus Calmette-Guérin. The Journal of Neuroscience. 2009;29(13):4200–4209. doi: 10.1523/JNEUROSCI.5032-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Painsipp E, Köfer MJ, Farzi A, Dischinger US, Sinner F, Herzog H, Holzer P. Neuropeptide Y and peptide YY protect from weight loss caused by Bacille Calmette-Guérin in mice. British Journal of Pharmacology. 2013;170(5):1014–1026. doi: 10.1111/bph.12354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Platt B, Schulenberg J, Klee N, Nizami M, Clark J. A depressive phenotype induced by Bacille Calmette Guérin in 'susceptible' animals: Sensitivity to antidepressants. Psychopharmacology. 2013;226(3):501–513. doi: 10.1007/s00213-012-2923-6. [DOI] [PubMed] [Google Scholar]
  18. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2012. http://www.R-project.org. [Google Scholar]
  19. SAS Institute Inc. SAS/STAT® 13.1 User's Guide. Cary, NC: SAS Institute Inc.; 2013. [Google Scholar]
  20. Serão NVL, González-Peña D, Beever JE, Bollero GA, Southey BR, Faulkner DB, Rodriguez-Zas SL. Bivariate genome-wide association analysis of the growth and intake components of feed efficiency. PloS One. 2013;8(10):e78530. doi: 10.1371/journal.pone.0078530. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Stearns TM, Beever JE, Southey BR, Ellis M, McKeith FK, Rodriguez-Zas SL. Evaluation of approaches to detect quantitative trait loci for growth, carcass, and meat quality on swine chromosomes 2, 6, 13, and 18. II. Multivariate and principal component analyses. Journal of Animal Science. 2005;83(11):2471–2481. doi: 10.2527/2005.83112471x. [DOI] [PubMed] [Google Scholar]
  22. Stevens JP. Applied multivariate statistics for the social sciences. Fifth Edition. New York, NY: Routledge, Taylor & Francis Group; 2009. [Google Scholar]
  23. Strekalova T, Couch Y, Kholod N, Boyks M, Malin D, Leprince P, Steinbusch HM. Update in the methodology of the chronic stress paradigm: internal control matters. Behavioral and Brain Functions. 2011;7:9. doi: 10.1186/1744-9081-7-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Vijaya Kumar K, Rudra A, Sreedhara MV, Siva Subramani T, Prasad DS, Das ML, Murugesan S, Yadav R, Trivedi RK, Louis JV, Li YW, Bristow LJ, Naidu PS, Vikramadithyan RK. Bacillus Calmette-Guerin vaccine induces a selective serotonin reuptake inhibitor (SSRI)-resistant depression like phenotype in mice. Brain, Behavior and, Immunity. 2014 doi: 10.1016/j.bbi.2014.06.205. pii: S0889-1591(14)00390-0. [DOI] [PubMed] [Google Scholar]
  25. Walker JR, Graff LA, Dutz JP, Bernstein CN. Psychiatric disorders in patients with immune-mediated inflammatory diseases: prevalence, association with disease activity, and overall patient well-being. J Rheumatol Suppl. 2011;88:31–35. doi: 10.3899/jrheum.110900. [DOI] [PubMed] [Google Scholar]
  26. Zuur AF, Ieno EN, Smith GM. Analysing ecological data. New York: Springer; 2007. [Google Scholar]

RESOURCES