Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Jun 23.
Published in final edited form as: Psychoneuroendocrinology. 2007 Nov 26;33(1):108–117. doi: 10.1016/j.psyneuen.2007.10.007

Interleukin-1 receptor type 1-deficient mice fail to develop social stress-associated glucocorticoid resistance in the spleen

Harald Engler a,b,*, Michael T Bailey a, Andrea Engler a,b, LaTonia M Stiner-Jones a, Ning Quan a, John F Sheridan a,c,d
PMCID: PMC2701260  NIHMSID: NIHMS117843  PMID: 18037570

Summary

Frequent or chronic stress as a result of repeated or persistent exposure to social challenges has been shown to affect the glucocorticoid (GC) responsiveness of immune cells in mice. Lipopolysaccharide-stimulated splenocytes of mice that were repeatedly subjected to social disruption were less sensitive to the anti-inflammatory actions of GC as evident from an increased production of pro-inflammatory cytokines and enhanced cell survival. The development of functional GC resistance was accompanied by the accumulation of GC-insensitive CD11b+ cells in the spleen. These cells were shown to exhibit impaired nuclear translocation of the GC receptor and lack of GC-induced suppression of NF-κB. Similar impairments in GC receptor function have been reported after in vitro treatment of various cell lines with interleukin (IL)-1. The aim of this study was to elucidate whether IL-1 is a critical factor for the development of GC resistance in socially stressed mice. In the first experiment, we investigated if repeated social stress alters plasma levels and tissue gene expression of IL-1α and IL-1β. It revealed that recurrent stressor exposure significantly increased splenic and hepatic mRNA expression and the plasma protein level of IL-1β, and hepatic mRNA expression of IL-1α. In the second experiment, IL-1 receptor type 1 (IL1R1)-deficient mice were subjected to the stressor and both the tissue distribution of CD11b+ cells and the GC sensitivity of the splenocytes were compared to wildtype mice. Mice lacking the IL1R1 exhibited adrenal hypertrophy, thymic involution, and elevated serum corticosterone levels in response to the stressor but did not show splenic accumulation of CD11b+ cells and failed to develop GC resistance. These findings suggest that IL-1 plays a critical role in the development of the social stress-associated GC resistance in the murine spleen.

Keywords: Proinflammatory cytokines, Glucocorticoid resistance, Social stress, Spleen, Lipopolysaccharide, CD11b

1. Introduction

Endogenous glucocorticoids (GC) play an important role in the control of the inflammatory response (Barnes, 1998). The production and the secretion of pro-inflammatory cytokines such as interleukin (IL)-1β and tumor necrosis factor (TNF)-α by activated immune cells trigger the activation of the hypothalamic–pituitary–adrenal (HPA) axis, leading to the release of GC from the adrenal cortex (Besedovsky and del Rey, 1996; Turnbull and Rivier, 1999). In turn, adrenal corticosteroids inhibit the expression of pro-inflammatory mediators, chemokines and cell adhesion molecules, and consequently limit the trafficking of immune cells to the site of inflammation. This bidirectional communication between the immune system and the HPA axis represents an important regulatory mechanism that protects the body from the detrimental consequences of an unconstrained inflammatory response. Failure of GC to suppress inflammatory processes due to a decrease in the corticosteroid sensitivity of target cells may promote the development and progression of various medical illnesses including inflammatory/infectious diseases, autoimmune diseases, and cancer (Bailey et al., 2003; Bamberger et al., 1996; del Rey and Besedovsky, 2000; Raison and Miller, 2003).

Severe stress, as a result of repeated or persistent exposure to behavioral or psychological challenges, has been shown to impair the anti-inflammatory capacities of GC hormones (Johnson et al., 2004; O’Connor et al., 2003a; Quan et al., 2001; Reber et al., 2006; Sheridan et al., 2000). Previous studies have demonstrated that repeated social stress in mice is associated with decreased GC sensitivity of immune cells and excessive inflammation (Avitsur et al., 2001; Engler et al., 2005; Johnson et al., 2004; Merlot et al., 2004; Sonoda et al., 2005; Stark et al., 2001). For example, lipopolysaccharide (LPS)-stimulated splenocytes from mice that repeatedly experienced acute encounters with an aggressive conspecific exhibited considerably higher cell survival in the presence of corticosterone (CORT) and showed greatly enhanced production of pro-inflammatory cytokines compared to splenocyte cultures from non-stressed controls (Avitsur et al., 2005, 2003; Merlot et al., 2004; Stark et al., 2001). The decrease in splenic GC responsiveness was only observed after stimulation with LPS whereas unstimulated cells did not differ in their ex vivo response to CORT. Importantly, the socially defeated mice were also more susceptible to endotoxic shock and, after in vivo challenge with LPS, showed significantly higher IL-1β and TNF-α expression in the spleen (Quan et al., 2001). The stress-associated decrease in the GC sensitivity of LPS-stimulated splenocytes was dependent on the presence of GC-insensitive CD11b+ myeloid cells, which have been shown to be mobilized from the bone marrow (Engler et al., 2004, 2005). Removal of these cells prior to cell culture abolished the stressor-induced effects on both cell survival and cytokine production (Stark et al., 2001). Molecular studies revealed an impaired nuclear translocation of the GC receptor (GCR) and a lack in transcriptional suppression of NF-κB in CD11b+ splenocytes of socially stressed mice (Quan et al., 2003).

Several lines of evidence suggest that the development of GC resistance might be related to the action of pro-inflammatory mediators. Molecular studies on the in vitro effects of pro-inflammatory cytokines on GCR function have demonstrated that both IL-1 and TNF-α can attenuate GCR translocation and GCR-mediated gene transcription in various cell lines (Pariante et al., 1999; Raddatz et al., 2001; Wang et al., 2004). Furthermore, in vitro treatment of human lymphoid cells with IL-1 or TNF-α was shown to cause the accumulation of the dominant negative beta-isoform of the GCR which was correlated with decreased GC sensitivity of these cells (Webster et al., 2001). In addition, various types of experimental stressors have been shown to induce gene expression and protein secretion of IL-1β in humans (Brydon et al., 2005; Heinz et al., 2003) and laboratory animals (Johnson et al., 2005b; Jung et al., 2000; O’Connor et al., 2003b). Thus, the present study was designed to elucidate whether the pro-inflammatory cytokine IL-1 is involved in the stress-associated development of functional GC resistance in the murine spleen. In the first experiment, the impact of repeated social disruption (SDR) stress on plasma protein concentration and tissue gene expression of IL-1α and IL-1β were determined in wildtype (WT) mice. In the second experiment, IL-1 receptor type 1-deficient (IL-1R1−/−) mice were subjected to six cycles of SDR and both the tissue distribution of CD11b+ cells and the ex vivo GC sensitivity of the splenocytes were compared to WT mice. Given that repeated exposure to the social stressor would result in increased IL-1 production and secretion, we hypothesized that mice lacking a functional IL-1 receptor type 1 would not develop functional GC resistance in the spleen.

2. Materials and methods

2.1. Experimental mice

Male C57BL/6 and IL-1R1−/− mice at the age of 6–8 weeks were used in this study. The IL-1R1−/− mice were descendants of breeding pairs originally obtained from the Jackson Laboratory (Bar Harbor, ME, USA). They were shown to exhibit a normal phenotype without overt abnormalities in the immune and hematopoietic systems and their response to LPS challenge was equivalent to WT mice (Glaccum et al., 1997). WT mice of the C57BL/6 background strain were purchased from Charles River Laboratories (Wilmington, MA, USA). All animals were housed 3 per cage in a facility accredited by the American Association for the Accreditation of Laboratory Animal Care and were maintained under environmentally controlled conditions on a 12:12-h light/dark cycle with ad libitum access to food and water. Mice were allowed to acclimate to the new surroundings for 2 weeks before initiation of any experimental procedure. All procedures were approved by the Institutional Laboratory Animal Care and Use Committee (ILACUC) of the Ohio State University.

2.2. Social stress procedure

The SDR procedure was previously described elsewhere (Avitsur et al., 2001). The stress paradigm is based on the repeated social defeat experience of group-housed male mice in their home cage, which is experimentally induced by daily confrontations with an aggressive intruder mouse. Each day at the beginning of the active phase, an intruder mouse was transferred into a cage with the group-housed experimental mice (residents). The intruder attacked the unfamiliar mice within the first minutes of the confrontation and initiated the display of submissive behavior (e.g., flight, defensive upright posture, retreat, crouch) in the residents. Intruders that did not display offensive agonistic behavior vs. the resident mice were replaced by another intruder. After 2 h of confrontation, the intruder was removed from the resident’s cage and returned to its home cage. The residents were carefully inspected for bite injuries after each confrontation cycle, and mice with severe wounds (<5%) were removed from the study. Experimental groups underwent this procedure for 6 consecutive days and their physiological response was compared to home cage controls housed in a separate room.

2.3. Intruder mice

Male C57BL/6 mice were used as intruder mice for the social stress procedure. These mice were older than the experimental animals and were individually housed over a period of several weeks to induce high levels of offensive agonistic behavior vs. unfamiliar mice. Intruder mice were pre experimentally screened in dyadic confrontations for their aggressiveness and only animals that consistently displayed offensive agonistic behavior were used in the experiments.

2.4. Blood and tissue collection

For the first experiment, WT mice were sacrificed immediately after the last confrontation cycle. Blood samples were taken by cardiac puncture and collected in EDTA-coated tubes. Spleen and liver were removed and snap-frozen in liquid nitrogen. The separated plasma and the tissue samples were stored at −80 °C until analysis. For the second experiment, WT and IL-1R1−/− mice were sacrificed 12 h after the last stressor exposure. Blood was taken by cardiac puncture and collected in EDTA-coated tubes. Spleen, femurs, and tibias of each mouse were aseptically removed and transferred to sterile tubes containing Hanks’ balanced salt solution (HBSS; Invitrogen, Carlsbad, CA, USA). Thymus and adrenal mass were determined as an indirect measure for chronic adrenocortical activation (Engler et al., 2005; Engler and Stefanski, 2003). In an additional experiment, blood was collected from the retro-orbital plexus to measure serum corticosterone levels under control conditions and immediately after the last SDR cycle.

2.5. Plasma protein levels of IL-1α and IL-1β

Plasma concentrations of IL-1α and IL-1β were measured with commercially available ELISA kits according to the manufacturer’s instructions (BioSource, Camarillo, CA, USA). The detection limits of the assays were 4 pg/ml (IL-1α) and 7 pg/ml (IL-1β), respectively, and the intra-assay coefficient of variation was <5% for both cytokines. All samples were measured in one assay.

2.6. Tissue gene expression of IL-1α and IL-1β

Total RNA from the spleen and the liver was isolated using Trizol reagent (Invitrogen). Single-strand cDNA was generated by reverse transcription (RT) using the Promega Reverse Transcription System with AMV reverse transcriptase and random hexamer primers (Promega, Madison, WI, USA). The reaction mixture (40 µl) contained 2 µg total RNA, 4 µl of 10 × RT buffer, 5mM MgCl2, 1mM each dNTP, 1 µg random hexamer primers, 40 U recombinant RNasin RNase inhibitor, and 30 U AMV Reverse Transcriptase. After incubation for 10 min at 25 °C, RT was carried out for 60 min at 42 °C, followed by AMV inactivation for 5 min at 95 °C. The synthesized cDNA was stored at −20 °C. Real-time PCR was performed with an ABI Prism 7000 Sequence Detection System (Applied Biosystems; Perkin-Elmer, Norwalk, CT, USA) using TaqMan Universal PCR Master Mix containing ROX as passive reference, 900 nM forward and reverse primer and 50 nM probe in a total volume of 25 µl with standard cycling conditions of 2 min at 50 °C, 10 min at 95 °C and 40 cycles of 15 s at 95 °C and 1 min at 60 °C. Primers and probes were selected using Primer Express software (Applied Biosystems) and were as follows: IL-1α (forward: 5′-GTC GGC AAA GAA ATC AAG ATG G-3′, reverse: 5′-TCA ATG GCA GAA CTG TAG TCT TCG-3′, probe: 5′-6-FAM-CCT GAC TTG TTT GAA GAC CTA AAG AAC TGT TAC AGT GA-TAMRA-3′), IL-1β (forward: 5′-GGC CTC AAA GGA AAG AAT CTA TAC C-3′, reverse: 5′-GTA TTG CTT GGG ATC CAC ACT CT-3′, probe: 5′-6-FAM-ATG AAA GAC GGC ACA CCC ACC CTG-TAMRA-3′) and 18S rRNA (forward: 5′-CGG CTA CCA CAT CCA AGG AA-3′, reverse: 5′-GCT GGA ATT ACC GCG GCT-3′, probe: 5′-VIC-TGC TGG CAC CAG ACT TGC CCT C-TAMRA-3′). Relative quantitation was performed according to Pfaffl (2001).

2.7. Leukocyte isolation

The epiphyses of the bones were cut off and the marrow was flushed out with cold HBSS. Single cell suspensions of the bone marrow were prepared by repeated passage of the tissue through a 23 gauge needle. Splenocyte suspensions were obtained by processing the tissue in cold HBSS in a paddle blender (Stomacher 80 Biomaster, Seward, London, UK). Red blood cells were removed using ACK lysis buffer (0.16M NH4Cl, 10mM KHCO3, 0.13mM EDTA, pH 7.2). The cells were then washed in HBSS containing 10% FBS, filtered through a 70-µm nylon cell strainer and resuspended in cell culture medium (RPMI 1640 supplemented with 10% FBS, 10mM HEPES, 1.5mM l-glutamine, 8.9mM NaHCO3, and 50 µg/ml gentamicin). Cell concentrations were determined with an automatic cell counter (Coulter Z2; Beckman-Coulter, Miami, FL, USA).

2.8. Glucocorticoid sensitivity of the splenocytes

The splenocytes were adjusted to a final concentration of 2.5 × 106 cells/ml in cell culture medium. The cell suspensions were stimulated with LPS from Escherichia coli (serotype O111:B4; Sigma-Aldrich, St. Louis, MO, USA) or remained untreated to assess background activity. To determine the GC sensitivity of unstimulated and LPS-stimulated cells, aliquots of each sample were treated with different amounts of corticosterone (CORT; Sigma-Aldrich) including both physiological and pharmacological concentrations of hormone. The final CORT concentrations were 0.005, 0.05, 0.1, 0.5, and 5 µM, respectively. The final LPS concentration was 1 µg/ml. Triplicates of each treatment were transferred to 96-well flat-bottom microtiter plates and incubated for 48 h in a humidified incubator (37 °C, 5% CO2). The cell viability in the splenocyte cultures was determined with a commercially available assay (CellTiter 96; Promega). To account for differences in background activity, the mean absorbance of the triplicate set of wells with unstimulated cells for a given treatment was subtracted from the mean absorbance of the three corresponding LPS-stimulated wells. As an index for the GC sensitivity of the cell cultures, an effective dose (ED) 50 value for each individual dose–response curve was calculated. The ED50 reflects the specific CORT concentration that is required to suppress the cell viability to 50% of the viability in the same sample without hormone. To calculate the ED50, exponential functions with a coefficient of determination of r2≥0.98 were used.

2.9. Blood and tissue distribution of CD11b+ cells

Total leukocyte counts in the blood, the bone marrow, and the spleen were determined with an automatic cell counter. For the identification and quantification of the CD11b+ cells, aliquots of the blood (20 µl) and the tissue cell suspensions (2.5 × 105 cells) were incubated with FITC-conjugated anti-mouse CD45 (clone 30-F11), PE-conjugated anti-mouse Pan-NK (clone DX-5), PerCP-Cy5.5-conjugated anti-mouse Gr-1 (clone RB6-8C5), and APC-conjugated anti-mouse CD11b (clone M1/70) for 45 min at 4 °C. All monoclonal antibodies were obtained from BD Pharmingen (San Diego, CA, USA). Antibody labeling was performed by a standard lyse-wash procedure using FACS lysing solution (BD Immunocytometry Systems, San Jose, CA, USA) and supplemented PBS (Dulbecco’s PBS; 2% FBS, 0.1% NaN3). Ten thousand cells from each sample were analyzed on a dual-laser flow cytometer (FACSCalibur; BD Immunocytometry Systems) using CellQuest Pro (Version 4.0.2) and Attractors (Version 3.1.0) software. Nucleated red blood cells were excluded from the analysis based on the lack of CD45 expression (Fornas et al., 2002). Matched isotype controls were used to set negative staining criteria.

2.10. Serum corticosterone concentration

Serum levels of CORT were measured using a commercially available radioimmunoassay according to the manufacturer’s instructions (Corticosterone 125I RIA; MP Biomedicals, Costa Mesa, CA, USA). The cross-reactivity of the antiserum with other relevant hormones and metabolites was <0.5%. The assay sensitivity was 7.7 ng/ml and the intra-assay variation was 4.4%. All samples were measured in one assay.

2.11. Statistical analysis

The Shapiro–Wilk test was used to assess the distribution of the data. Protein and mRNA expression data of IL-1α and IL-1β were analyzed using Mann–Whitney U-test for non-parametric comparisons. Organ weights, CD11b+ cell distribution data, and CORT levels were analyzed by two-factor (stress × genotype) ANOVA. Glucocorticoid sensitivity data were analyzed by repeated measure ANOVA with ‘CORT concentration’ as repeated measures within-subject factor and ‘stress’ and ‘genotype’ as between-subjects factors. Post hoc comparisons using Bonferroni-corrected t-tests or Tukey’s HSD test were performed when appropriate. Data were analyzed using SPSS for Windows (Version 14.0.2; SPSS Inc., Chicago, IL, USA). The level of significance was set at p<0.05. All results are expressed as mean ± S.E.M.

3. Results

3.1. Plasma protein and tissue mRNA levels of IL-1α and IL-1β

The impact of SDR stress on the gene expression and the protein levels of IL-1 were determined in WT mice. Repeated exposure of the experimental mice to the social stressor had marked effects on both the plasma protein level and the tissue gene expression of IL-1 (Figure 1). Immediately after the last confrontation cycle, the plasma concentration (U = 0, p = 0.004) as well as the mRNA expression level of IL-1β in the spleen (U = 3, p = 0.047) and the liver (U = 0, p = 0.004) were significantly elevated in stressed mice compared to controls. In addition, the mRNA expression level of IL-1α in the liver was significantly higher in mice that repeatedly underwent SDR (U = 4, p = 0.025).

Figure 1.

Figure 1

Open in a new tab

Plasma protein concentration and tissue mRNA expression of IL-1α and IL-1β in control mice and in mice that were subjected to six cycles of social disruption (SDR). Mann–Whitney U-test; *p<0.05, **p<0.001; n = 6 per group.

3.2. Adrenal, thymus, and spleen mass

Repeated exposure to the acute social stressor resulted in adrenal hypertrophy, thymic involution, and splenic enlargement in both WT and IL-1R1−/− mice (Figure 2). Two-factor ANOVA revealed significant main effects of ‘stress’ for all three tissues (adrenals: F[3,20] = 25.73, thymus: F[3,20] = 115.24, spleen: F[3,20] = 28.94; p<0.001). In addition, there was a significant main effect of ‘genotype’ for adrenal mass (F[3,20] = 6.68, p<0.05) as well as a significant ‘stress × genotype’ interaction for thymus mass (F[3,20] = 5.25, p<0.05). Post hoc comparisons showed that WT and IL-1R1−/− SDR mice had a significantly lower thymus mass and significantly heavier adrenals and spleens than the controls (p<0.05).

Figure 2.

Figure 2

Open in a new tab

Social stress caused adrenal hypertrophy, thymic involution, and splenic enlargement in wildtype (WT) and IL-1 receptor type 1-deficient (IL-1R1−/−) mice. Organ masses from control mice and from mice that were subjected to six cycles of social disruption (SDR) are shown. Two-factor ANOVA; *p<0.05 vs. control groups; n = 6 per group.

3.3. Serum corticosterone concentration

To account for possible differences in the CORT secretion between WT and IL-1R1−/− mice, serum CORT levels were determined in control animals and in SDR mice of both strains immediately after the last confrontation cycle. WT and IL-1R1−/− mice exhibited comparable basal CORT levels and showed similar serum CORT increases in response to the stressor (Table 1). Two-factor ANOVA revealed a significant main effect of ‘stress’ (F[3,20] = 49.78; p<0.001) but there was neither a significant main effect of ‘genotype’ (F[3,20] = 0.13; n.s.) nor a ‘stress × genotype’ interaction (F[3,20] = 0.29; n.s.).

Table 1.

Serum corticosterone levels in wildtype (WT) and IL-1 receptor type 1-deficient (IL-1R1−/−) mice under control conditions and immediately after the last confrontation cycle (SDR).

Control SDR
WT 52 ± 13 319 ± 50*
IL-1R1−/− 58 ± 14 288 ± 81*
Open in a new tab

Wildtype and IL-1R1−/− mice exhibited comparable basal CORT levels and showed similar serum CORT increases in response to the stressor. Data are expressed as ng/ml. n = 12 per strain.

*

p<0.05 vs. controls.

3.4. Glucocorticoid sensitivity of the splenocytes

Daily confrontations with an unfamiliar, aggressive intruder mouse over a period of 6 consecutive days induced functional GC resistance in the spleens of WT mice but not in IL-1R1−/− mice (Figure 3). Mean cell viabilities of the LPS-stimulated splenocytes in presence of 5 µM CORT were 30 ± 3% (WT controls), 70 ± 6% (WT SDR), 29 ± 1% (IL-1R1−/− controls) and 37 ± 5% (IL-1R1−/− SDR), respectively, compared to the same samples in the absence of CORT. The calculated ED50 were 0.325 µM (WT controls), 0.210 µM (IL-1R1−/− controls), and 0.411 µM (IL-1R1−/− SDR). Calculation of the ED50 for the WT SDR mice was not feasible since none of the CORT concentrations used decreased the cell viability to 50%. Therefore, the ED50 for this group was considered >5 µM. Repeated measure ANOVA on the cell viability data of the GC-treated splenocyte cultures revealed significant main effects of ‘stress’ (F[1,20] = 18.67, p<0.001), genotype (F[1,20] = 8.99, p<0.01) and ‘CORT concentration’ (F[5,100] = 224.06, p<0.001) as well as a significant interaction of ‘stress × genotype × CORT concentration’ (F[5,100] = 4.93, p<0.001). Post hoc comparisons for each hormone concentration showed that the cell viability in splenocytes of WT SDR mice was significantly higher in the three highest CORT concentrations than in all other groups (0.1 µM CORT: F[3,20] = 17.41, 0.5 µM CORT: F[3,20] = 16.21, 5 µM CORT: F[3,20] = 19.73; p<0.001).

Figure 3.

Figure 3

Open in a new tab

Social stress caused functional glucocorticoid resistance in wildtype (WT) but not in IL-1 receptor type 1-deficient (IL-1R1−/−) mice. Ex vivo glucocorticoid sensitivity of splenocytes from control mice and from mice that were subjected to six cycles of social disruption (SDR). Twelve hours after the last stress cycle, splenocytes were isolated and cultured with LPS (1 µg/ml) in the presence of various concentrations of CORT. Data are presented as percentage of cell viability in the absence of CORT. Repeated measure ANOVA; *p<0.05 vs. all other groups; n = 6 per group.

3.5. Blood and tissue distribution of CD11b+ cells

The distribution of CD11b+ leukocytes in the bone marrow (BM), the blood (BL) and the spleen (SPL) of WTand IL-1R1−/− mice was differentially affected by the social stressor (Figure 4). Two-factor ANOVA revealed for the percentages and the absolute numbers of CD11b+ cells in the three compartments significant main effects of ‘stress’ (percentages BM: F[3,20] = 41.40; BL: F[3,20] = 49.80; SPL: F[3,20] = 43.81, all p<0.001; absolute numbers BM: F[3,20] = 6.83, p<0.05; BL: F[3,20] = 73.15; p<0.001; SPL: F[3,20] = 31.23, p<0.001), ‘genotype’ (percentages BM: F[3,20] = 42.82, p<0.001; BL: F[3,20] = 11.04; p<0.01; SPL F[3,20] = 30.65, p<0.001; absolute numbers BM: F[3,20] = 12.62, p<0.01; BL: F[3,20] = 17.92; p<0.001; SPL F[3,20] = 9.80, p<0.01), and significant ‘stress × genotype’ interactions (percentages BM: F[3,20] = 22.28; BL: F[3,20] = 17.30; SPL: F[3,20] = 25.33, all p<0.001; absolute numbers BM: F[3,20] = 9.61, p<0.01; BL: F[3,20] = 20.72; p<0.001; SPL: F[3,20] = 14.26, p<0.001). Post hoc comparisons demonstrated that the relative as well as the absolute numbers of CD11b+ cells were significantly higher in the bone marrow, blood, and spleen of WT SDR mice compared to the other three groups (p<0.05).

Figure 4.

Figure 4

Open in a new tab

Social stress differentially affected the distribution of CD11b+ cells in the bone marrow (BM), the blood (BL), and the spleen (SPL) of wildtype (WT) and IL-1 receptor type 1-deficient (IL-1R1−/−) mice. Absolute numbers (left) and percentages (right) of CD11b+ cells in control mice and in mice that were subjected to six cycles of social disruption (SDR) are shown. Two-factor ANOVA; *p<0.05 vs. all other groups, #p<0.05 vs. control groups; n = 6 per group.

4. Discussion

Psychological stress, stress-related diseases (e.g., posttraumatic stress disorder) and psychiatric disorders (e.g., major depression) have been shown to be associated with alterations in the GC sensitivity of immune cells (Bauer et al., 2003; de Kloet et al., 2007). In all these conditions, elevated plasma levels of IL-1 were found as well, suggesting a causal link between increased IL-1 production and the development of GC resistance (Spivak et al., 1997; Heinz et al., 2003; Pariante and Miller, 2001; Raison et al., 2006; Rohleder et al., 2003; Tucker et al., 2004). However, these data were solely correlative in nature.

In the present study, we demonstrated that the tissue gene expression and the plasma protein concentrations of the pro-inflammatory cytokines IL-1α and IL-1β were largely affected in mice repeatedly subjected to SDR stress. Immediately after the last confrontation cycle, plasma protein and mRNA expression levels of IL-1β in the spleen and the liver were significantly increased compared to controls. In addition, the mRNA expression level of IL-1α in the liver was elevated. These observations are in accordance with previous studies demonstrating that severe stress is associated with an increase in IL-1 production (Johnson et al., 2005b; O’Connor et al., 2003b). However, the factors triggering the stressor-induced secretion of IL-1 remain unknown. One possibility could be the activation of innate immune responses by opportunistic pathogens belonging to the host’s own microflora. Previous data from our laboratory have shown that chronic stress can induce the translocation of cutaneous and gastrointestinal microflora to secondary lymphoid organs (Bailey et al., 2006). Indigenous bacteria entering the host can initiate the secretion of proinflammatory cytokines via activation of pattern recognition receptors. However, due to the low levels of bacteria found in organs of stressed mice, local rather than a systemic increase in IL-1 would be more likely. An alternative explanation for the increase of IL-1 in the circulation, the spleen and the liver of stressed mice could be the activation of the immune system by endogenous danger signals (e.g., ATP, heat-shock proteins) which derive from damaged or dead cells (Johnson and Fleshner, 2006; Matzinger, 2002). These danger signals can activate caspase-1 through the NALP3-inflammasome, leading to the cleavage and secretion of IL-1β (Mariathasan and Monack, 2007; Ogura et al., 2006). Interestingly, recent studies have shown that pre-treatment of experimental animals with adrenergic receptor antagonists attenuates the stress-associated release of both heat-shock proteins and IL-1 suggesting that catecholamines could mediate the stressor-induced increase in pro-inflammatory cytokines (Blandino et al., 2006; Johnson et al., 2005a, b).

Repeated exposure to the social stressor caused a significant decrease in the GC sensitivity of splenocytes from WT mice whereas mice lacking the IL-1 receptor type 1 did not develop functional GC resistance. Importantly, this effect was not related to strain differences in the adrenocortical response. Both WT and IL-1R1−/− SDR mice showed significant increases in serum CORT levels, adrenal hypertrophy, and thymic atrophy, which is in line with previous reports demonstrating that IL-1R1−/− and WT mice exhibit equivalent increases in plasma CORT levels when exposed to moderate and severe stressors (Goshen et al., 2003; Wolf et al., 2007).

The GC sensitivity data of the IL-1R1−/− mice indicate that IL-1R1-mediated signaling is a critical factor for the stress-associated development of functional GC resistance in the murine spleen. Comparison of the distribution patterns of CD11b+ myeloid cells in the bone marrow, blood and spleen of WT and IL-1R1−/− SDR mice suggest that this was related to a lack in the mobilization of CD11b+ cells from the bone marrow. IL-1β stimulates the expression and secretion of myelopoietic cytokines (e.g., colony-stimulating factors) in stromal cells of the bone marrow and modulates the expression of adhesion molecules, resulting in the accelerated release of myeloid cells from the marrow and their migration to lymphoid tissues (Dinarello, 1996). We have previously demonstrated that SDR was associated with an increase in the expression of granulocyte-macrophage colony-stimulating factor (GM-CSF) in the bone marrow, enhanced myelopoiesis, and a massive accumulation of CD11b+ cells in the blood and the spleen (Engler et al., 2004, 2005). Since IL-1R1−/− mice subjected to the stressor did not show enhanced myelopoiesis in the bone marrow nor substantial increases of CD11b+ cell numbers in the blood and the spleen, we conclude that IL-1 was mediating the stress-associated mobilization and redistribution of GC-insensitive cells to the spleen which ultimately resulted in a decreased GC sensitivity of this tissue.

The protein and the mRNA expression profiles of the two molecular forms of IL-1 in the plasma and the tissues of stressed mice suggest that IL-1β might be the main trigger for the mobilization of the CD11b+ cells and for the development of functional GC resistance. However, using IL-1R1−/− mice, we cannot conclude that IL-1β was responsible for the stress-associated changes in cell distribution and GC sensitivity. Both IL-1α and IL-1β signal through the IL-1 type I receptor and in vitro studies have shown that IL-1α can inhibit GCR translocation and function as well (Pariante et al., 1999). But in contrast to IL-1α, where only increased hepatic mRNA levels were found in SDR mice, IL-1β was elevated in all three compartments analyzed including plasma protein levels. Moreover, recent studies with mice bearing tumors that overexpress and secrete IL-1β exhibited similar increases of CD11b+ cells in the blood and spleen, suggesting that the mobilization of CD11b+ cells in SDR mice is indeed caused by IL-1β (Song et al., 2005). However, final conclusions can only be drawn from additional experiments with IL-1α and IL-1β-deficient mice.

IL-1 has been shown to inhibit GCR signaling in vitro as demonstrated by decreased GCR translocation and diminished activation of relevant GCR-induced genes (Pariante et al., 1999). Several signaling pathways seem to be involved in the disruption of the GCR translocation and function, leading to the development of GC resistance (Pace et al., 2007). In particular, the mitogen-activated protein kinase (MAPK) pathways have recently received a lot of attention. Through binding to its specific receptor, IL-1 activates MAPK pathways including the p38, the extracellular signal-related kinase (ERK), and the Jun N-terminal kinase (JNK) signal transduction pathways. Activation of p38, ERK amd JNK have been shown to inhibit GCR function via direct or indirect phosphorylation of the GCR which prevents its translocation and DNA binding. Impaired nuclear translocation of the GCR and lack in transcriptional suppression of NF-κB were found in CD11b+ splenocytes of SDR mice (Quan et al., 2003). Thus, based on the findings of the present study, we assume that SDR leads to IL-1-induced activation of MAPK pathways which impairs GCR function and ultimately leads to the development of GC resistance in these cells. Molecular studies on enriched CD11b+ cells from SDR and control mice are currently in progress to elucidate if SDR indeed results in the activation of the above-mentioned signal transduction pathways.

In conclusion, the present study demonstrates that the increase in IL-1 production during chronic social stress is an important factor contributing to the development of functional GC resistance in murine spleen. The stress-associated release of IL-1 appears to induce the generation of GC insensitive CD11b+ cells in the bone marrow and their accumulation in the spleen. These findings suggest that alterations in the GC sensitivity of immune cells from patients with stress-related diseases and psychiatric disorders might be related to the elevated plasma levels of IL-1 reported in these subjects. Future studies need to elucidate the factors triggering the stress-associated release of IL-1.

Acknowledgments

Role of the funding sources

Funding for this study was provided by the National Institutes of Health. The funding source had no involvement in study design, in the collection, analysis and interpretation of data, in the writing of the report, and in the decision to submit the paper for publication.

The authors thank Jacqueline Verity for excellent technical assistance. The study was supported by grants from the National Institute of Mental Health (RO1 MH46801-15) and the National Institute of Dental and Craniofacial Research (T32DE014320-06) to J.F.S.

Footnotes

Conflict of interest

All authors declare no conflict of interest.

References

  1. Avitsur R, Stark JL, Sheridan JF. Social stress induces glucocorticoid resistance in subordinate animals. Horm. Behav. 2001;39:247–257. doi: 10.1006/hbeh.2001.1653. [DOI] [PubMed] [Google Scholar]
  2. Avitsur R, Padgett DA, Dhabhar FS, Stark JL, Kramer KA, Engler H, Sheridan JF. Expression of glucocorticoid resistance following social stress requires a second signal. J. Leukoc. Biol. 2003;74:507–513. doi: 10.1189/jlb.0303090. [DOI] [PubMed] [Google Scholar]
  3. Avitsur R, Kavelaars A, Heijnen C, Sheridan JF. Social stress and the regulation of tumor necrosis factor-alpha secretion. Brain Behav. Immun. 2005;19:311–317. doi: 10.1016/j.bbi.2004.09.005. [DOI] [PubMed] [Google Scholar]
  4. Bailey M, Engler H, Hunzeker J, Sheridan JF. The hypothalamic–pituitary–adrenal axis and viral infection. Viral Immunol. 2003;16:141–157. doi: 10.1089/088282403322017884. [DOI] [PubMed] [Google Scholar]
  5. Bailey MT, Engler H, Sheridan JF. Stress induces the translocation of cutaneous and gastrointestinal microflora to secondary lymphoid organs of C57BL/6 mice. J. Neuroimmunol. 2006;171:29–37. doi: 10.1016/j.jneuroim.2005.09.008. [DOI] [PubMed] [Google Scholar]
  6. Bamberger CM, Schulte HM, Chrousos GP. Molecular determinants of glucocorticoid receptor function and tissue sensitivity to glucocorticoids. Endocr. Rev. 1996;17:245–261. doi: 10.1210/edrv-17-3-245. [DOI] [PubMed] [Google Scholar]
  7. Barnes PJ. Anti-inflammatory actions of glucocorticoids: molecular mechanisms. Clin. Sci. (London) 1998;94:557–572. doi: 10.1042/cs0940557. [DOI] [PubMed] [Google Scholar]
  8. Bauer ME, Papadopoulos A, Poon L, Perks P, Lightman SL, Checkley S, Shanks N. Altered glucocorticoid immunoregulation in treatment resistant depression. Psychoneuroendocrinology. 2003;28:49–65. doi: 10.1016/s0306-4530(02)00009-4. [DOI] [PubMed] [Google Scholar]
  9. Besedovsky HO, del Rey A. Immune–neuro–endocrine interactions: facts and hypotheses. Endocr. Rev. 1996;17:64–102. doi: 10.1210/edrv-17-1-64. [DOI] [PubMed] [Google Scholar]
  10. Blandino P, Jr, Barnum CJ, Deak T. The involvement of norepinephrine and microglia in hypothalamic and splenic IL-1β responses to stress. J. Neuroimmunol. 2006;173:87–95. doi: 10.1016/j.jneuroim.2005.11.021. [DOI] [PubMed] [Google Scholar]
  11. Brydon L, Edwards S, Jia H, Mohamed-Ali V, Zachary I, Martin JF, Steptoe A. Psychological stress activates interleukin-1β gene expression in human mononuclear cells. Brain Behav. Immun. 2005;19:540–546. doi: 10.1016/j.bbi.2004.12.003. [DOI] [PubMed] [Google Scholar]
  12. de Kloet CS, Vermetten E, Bikker A, Meulman E, Geuze E, Kavelaars A, Westenberg HG, Heijnen CJ. Leukocyte glucocorticoid receptor expression and immunoregulation in veterans with and without post-traumatic stress disorder. Mol. Psychiatry. 2007;12:443–453. doi: 10.1038/sj.mp.4001934. [DOI] [PubMed] [Google Scholar]
  13. del Rey A, Besedovsky HO. The cytokine–HPA axis circuit contributes to prevent or moderate autoimmune processes. Z. Rheumatol. 2000;59 Suppl. 2:II/31–II/35. doi: 10.1007/s003930070015. [DOI] [PubMed] [Google Scholar]
  14. Dinarello CA. Biologic basis for interleukin-1 in disease. Blood. 1996;87:2095–2147. [PubMed] [Google Scholar]
  15. Engler H, Stefanski V. Social stress and T cell maturation in male rats: transient and persistent alterations in thymic function. Psychoneuroendocrinology. 2003;28:951–969. doi: 10.1016/s0306-4530(02)00117-8. [DOI] [PubMed] [Google Scholar]
  16. Engler H, Bailey MT, Engler A, Sheridan JF. Effects of repeated social stress on leukocyte distribution in bone marrow, peripheral blood and spleen. J. Neuroimmunol. 2004;148:106–115. doi: 10.1016/j.jneuroim.2003.11.011. [DOI] [PubMed] [Google Scholar]
  17. Engler H, Engler A, Bailey MT, Sheridan JF. Tissue-specific alterations in the glucocorticoid sensitivity of immune cells following repeated social defeat in mice. J. Neuroimmunol. 2005;163:110–119. doi: 10.1016/j.jneuroim.2005.03.002. [DOI] [PubMed] [Google Scholar]
  18. Fornas O, Domingo JC, Marin P, Petriz J. Flow cytometric-based isolation of nucleated erythroid cells during maturation: an approach to cell surface antigen studies. Cytometry. 2002;50:305–312. doi: 10.1002/cyto.10158. [DOI] [PubMed] [Google Scholar]
  19. Glaccum MB, Stocking KL, Charrier K, Smith JL, Willis CR, Maliszewski C, Livingston DJ, Peschon JJ, Morrissey PJ. Phenotypic and functional characterization of mice that lack the type I receptor for IL-1. J. Immunol. 1997;159:3364–3371. [PubMed] [Google Scholar]
  20. Goshen I, Yirmiya R, Iverfeldt K, Weidenfeld J. The role of endogenous interleukin-1 in stress-induced adrenal activation and adrenalectomy-induced adrenocorticotropic hormone hypersecretion. Endocrinology. 2003;144:4453–4458. doi: 10.1210/en.2003-0338. [DOI] [PubMed] [Google Scholar]
  21. Heinz A, Hermann D, Smolka MN, Rieks M, Graf KJ, Pohlau D, Kuhn W, Bauer M. Effects of acute psychological stress on adhesion molecules, interleukins and sex hormones: implications for coronary heart disease. Psychopharmacology (Berlin) 2003;165:111–117. doi: 10.1007/s00213-002-1244-6. [DOI] [PubMed] [Google Scholar]
  22. Johnson JD, Fleshner M. Releasing signals, secretory pathways, and immune function of endogenous extracellular heat shock protein 72. J. Leukoc. Biol. 2006;79:425–434. doi: 10.1189/jlb.0905523. [DOI] [PubMed] [Google Scholar]
  23. Johnson RR, Storts R, Welsh TH, Jr, Welsh CJ, Meagher MW. Social stress alters the severity of acute Theiler’s virus infection. J. Neuroimmunol. 2004;148:74–85. doi: 10.1016/j.jneuroim.2003.11.009. [DOI] [PubMed] [Google Scholar]
  24. Johnson JD, Campisi J, Sharkey CM, Kennedy SL, Nickerson M, Fleshner M. Adrenergic receptors mediate stress-induced elevations in extracellular Hsp72. J. Appl. Physiol. 2005a;99:1789–1795. doi: 10.1152/japplphysiol.00390.2005. [DOI] [PubMed] [Google Scholar]
  25. Johnson JD, Campisi J, Sharkey CM, Kennedy SL, Nickerson M, Greenwood BN, Fleshner M. Catecholamines mediate stress-induced increases in peripheral and central inflammatory cytokines. Neuroscience. 2005b;135:1295–1307. doi: 10.1016/j.neuroscience.2005.06.090. [DOI] [PubMed] [Google Scholar]
  26. Jung BD, Kimura K, Kitamura H, Makondo K, Kanehira K, Saito M. Sympathetic activation of hepatic and splenic IL-1β mRNA expression during oscillation stress in the rat. J. Vet. Med. Sci. 2000;62:409–413. doi: 10.1292/jvms.62.409. [DOI] [PubMed] [Google Scholar]
  27. Mariathasan S, Monack DM. Inflammasome adaptors and sensors: intracellular regulators of infection and inflammation. Nat. Rev. Immunol. 2007;7:31–40. doi: 10.1038/nri1997. [DOI] [PubMed] [Google Scholar]
  28. Matzinger P. The danger model: a renewed sense of self. Science. 2002;296:301–305. doi: 10.1126/science.1071059. [DOI] [PubMed] [Google Scholar]
  29. Merlot E, Moze E, Dantzer R, Neveu PJ. Cytokine production by spleen cells after social defeat in mice: activation of T cells and reduced inhibition by glucocorticoids. Stress. 2004;7:55–61. doi: 10.1080/1025389042000208150. [DOI] [PubMed] [Google Scholar]
  30. O’Connor KA, Johnson JD, Hammack SE, Brooks LM, Spencer RL, Watkins LR, Maier SF. Inescapable shock induces resistance to the effects of dexamethasone. Psychoneuroendocrinology. 2003a;28:481–500. doi: 10.1016/s0306-4530(02)00035-5. [DOI] [PubMed] [Google Scholar]
  31. O’Connor KA, Johnson JD, Hansen MK, Wieseler Frank JL, Maksimova E, Watkins LR, Maier SF. Peripheral and central proinflammatory cytokine response to a severe acute stressor. Brain Res. 2003b;991:123–132. doi: 10.1016/j.brainres.2003.08.006. [DOI] [PubMed] [Google Scholar]
  32. Ogura Y, Sutterwala FS, Flavell RA. The inflammasome: first line of the immune response to cell stress. Cell. 2006;126:659–662. doi: 10.1016/j.cell.2006.08.002. [DOI] [PubMed] [Google Scholar]
  33. Pace TW, Hu F, Miller AH. Cytokine-effects on glucocorticoid receptor function: relevance to glucocorticoid resistance and the pathophysiology and treatment of major depression. Brain Behav. Immun. 2007;21:9–19. doi: 10.1016/j.bbi.2006.08.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pariante CM, Miller AH. Glucocorticoid receptors in major depression: relevance to pathophysiology and treatment. Biol. Psychiatry. 2001;49:391–404. doi: 10.1016/s0006-3223(00)01088-x. [DOI] [PubMed] [Google Scholar]
  35. Pariante CM, Pearce BD, Pisell TL, Sanchez CI, Po C, Su C, Miller AH. The proinflammatory cytokine, interleukin-1α, reduces glucocorticoid receptor translocation and function. Endocrinology. 1999;140:4359–4366. doi: 10.1210/endo.140.9.6986. [DOI] [PubMed] [Google Scholar]
  36. Pfaffl MW. A new mathematical model for relative quantification in real-time RT-PCR. Nucleic Acids Res. 2001;29:2002–2007. doi: 10.1093/nar/29.9.e45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Quan N, Avitsur R, Stark JL, He L, Shah M, Caligiuri M, Padgett DA, Marucha PT, Sheridan JF. Social stress increases the susceptibility to endotoxic shock. J. Neuroimmunol. 2001;115:36–45. doi: 10.1016/s0165-5728(01)00273-9. [DOI] [PubMed] [Google Scholar]
  38. Quan N, Avitsur R, Stark JL, He L, Lai W, Dhabhar F, Sheridan JF. Molecular mechanisms of glucocorticoid resistance in splenocytes of socially stressed male mice. J. Neuroimmunol. 2003;137:51–58. doi: 10.1016/s0165-5728(03)00042-0. [DOI] [PubMed] [Google Scholar]
  39. Raddatz D, Toth S, Schworer H, Ramadori G. Glucocorticoid receptor signaling in the intestinal epithelial cell lines IEC-6 and Caco-2: evidence of inhibition by interleukin-1β. Int. J. Colorectal Dis. 2001;16:377–383. doi: 10.1007/s003840100331. [DOI] [PubMed] [Google Scholar]
  40. Raison CL, Miller AH. When not enough is too much: the role of insufficient glucocorticoid signaling in the pathophysiology of stress-related disorders. Am. J. Psychiatry. 2003;160:1554–1565. doi: 10.1176/appi.ajp.160.9.1554. [DOI] [PubMed] [Google Scholar]
  41. Raison CL, Capuron L, Miller AH. Cytokines sing the blues: inflammation and the pathogenesis of depression. Trends Immunol. 2006;27:24–31. doi: 10.1016/j.it.2005.11.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Reber SO, Obermeier F, Straub HR, Falk W, Neumann ID. Chronic intermittent psychosocial stress (social defeat/overcrowding) in mice increases the severity of an acute DSS-induced colitis and impairs regeneration. Endocrinology. 2006;147:4968–4976. doi: 10.1210/en.2006-0347. [DOI] [PubMed] [Google Scholar]
  43. Rohleder N, Wolf JM, Kirschbaum C. Glucocorticoid sensitivity in humans—interindividual differences and acute stress effects. Stress. 2003;6:207–222. doi: 10.1080/1025389031000153658. [DOI] [PubMed] [Google Scholar]
  44. Sheridan JF, Stark JL, Avitsur R, Padgett DA. Social disruption, immunity, and susceptibility to viral infection. Role of glucocorticoid insensitivity and NGF. Ann. N. Y. Acad. Sci. 2000;917:894–905. doi: 10.1111/j.1749-6632.2000.tb05455.x. [DOI] [PubMed] [Google Scholar]
  45. Song X, Krelin Y, Dvorkin T, Bjorkdahl O, Segal S, Dinarello CA, Voronov E, Apte RN. CD11b+/Gr-1+ immature myeloid cells mediate suppression of T cells in mice bearing tumors of IL-1beta-secreting cells. J. Immunol. 2005;175:8200–8208. doi: 10.4049/jimmunol.175.12.8200. [DOI] [PubMed] [Google Scholar]
  46. Sonoda J, Chida Y, Sudo N, Kubo C. Social disruption stress exacerbates alpha-galactosylceramide-induced hepatitis in mice. Neuroimmunomodulation. 2005;12:375–379. doi: 10.1159/000091131. [DOI] [PubMed] [Google Scholar]
  47. Spivak B, Shohat B, Mester R, Avraham S, Gil-Ad I, Bleich A, Valevski A, Weizman A. Elevated levels of serum interleukin-1β in combat-related posttraumatic stress disorder. Biol. Psychiatry. 1997;42:345–348. doi: 10.1016/S0006-3223(96)00375-7. [DOI] [PubMed] [Google Scholar]
  48. Stark JL, Avitsur R, Padgett DA, Campbell KA, Beck FM, Sheridan JF. Social stress induces glucocorticoid resistance in macrophages. Am. J. Physiol. Regul. Integr. Comp. Physiol. 2001;280:R1799–R1805. doi: 10.1152/ajpregu.2001.280.6.R1799. [DOI] [PubMed] [Google Scholar]
  49. Tucker P, Ruwe WD, Masters B, Parker DE, Hossain A, Trautman RP, Wyatt DB. Neuroimmune and cortisol changes in selective serotonin reuptake inhibitor and placebo treatment of chronic posttraumatic stress disorder. Biol. Psychiatry. 2004;56:121–128. doi: 10.1016/j.biopsych.2004.03.009. [DOI] [PubMed] [Google Scholar]
  50. Turnbull AV, Rivier CL. Regulation of the hypothalamic–pituitary–adrenal axis by cytokines: actions and mechanisms of action. Physiol. Rev. 1999;79:1–71. doi: 10.1152/physrev.1999.79.1.1. [DOI] [PubMed] [Google Scholar]
  51. Wang X, Wu H, Miller AH. Interleukin-1α (IL-1α) induced activation of p38 mitogen-activated protein kinase inhibits glucocorticoid receptor function. Mol. Psychiatry. 2004;9:65–75. doi: 10.1038/sj.mp.4001339. [DOI] [PubMed] [Google Scholar]
  52. Webster JC, Oakley RH, Jewell CM, Cidlowski JA. Proinflammatory cytokines regulate human glucocorticoid receptor gene expression and lead to the accumulation of the dominant negative beta isoform: a mechanism for the generation of glucocorticoid resistance. Proc. Natl. Acad. Sci. USA. 2001;98:6865–6870. doi: 10.1073/pnas.121455098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Wolf G, Yirmiya R, Kreisel T, Goshen I, Weidenfeld J, Poole S, Shavit Y. Interleukin-1 signaling modulates stress-induced analgesia. Brain Behav. Immun. 2007;21:652–659. doi: 10.1016/j.bbi.2006.10.016. [DOI] [PubMed] [Google Scholar]

RESOURCES