Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Mar 9.
Published in final edited form as: J Psychiatr Res. 2016 Feb 6;76:59–65. doi: 10.1016/j.jpsychires.2016.02.001

Mitochondrial dysfunction and lipid peroxidation in rat frontal cortex by chronic NMDA administration can be partially prevented by lithium treatment

Helena K Kim a, Cameron Isaacs-Trepanier a, Nika Elmi a, Stanley I Rapoport b, Ana C Andreazza a,c,*
PMCID: PMC5843818  NIHMSID: NIHMS944840  PMID: 26894301

Abstract

Chronic N-methyl-d-aspartate (NMDA) administration to rats may be a model to investigate excitotoxicity mediated by glutamatergic hyperactivity, and lithium has been reported to be neuroprotective. We hypothesized that glutamatergic hyperactivity in chronic NMDA injected rats would cause mitochondrial dysfunction and lipid peroxidation in the brain, and that chronic lithium treatment would ameliorate some of these NMDA-induced alterations. Rats treated with lithium for 6 weeks were injected i.p. 25 mg/kg NMDA on a daily basis for the last 21 days of lithium treatment. Brain was removed and frontal cortex was analyzed. Chronic NMDA decreased brain levels of mitochondrial complex I and III, and increased levels of the lipid oxidation products, 8-isoprostane and 4-hydroxynonenal, compared with non-NMDA injected rats. Lithium treatment prevented the NMDA-induced increments in 8-isoprostane and 4-hydroxynonenal. Our findings suggest that increased chronic activation of NMDA receptors can induce alterations in electron transport chain complexes I and III and in lipid peroxidation in brain. The NMDA-induced changes may contribute to glutamate-mediated excitotoxicity, which plays a role in brain diseases such as bipolar disorder. Lithium treatment prevented changes in 8-isoprostane and 4-hydroxynonenal, which may contribute to lithium's reported neuroprotective effect and efficacy in bipolar disorder.

Keywords: NMDA, Lipid peroxidation, Mitochondria, Excitotoxicity, Lithium

1. Introduction

Excessive glutamate signaling can induce neuronal dysfunction and cell death by producing excitotoxicity, a process that is characterized by a large influx of calcium, resulting in a cascade of events involving disruption of calcium-dependent cellular pathways, mitochondrial dysfunction, production of oxidative stress and activation of apoptosis (Plitman et al., 2014).

Glutamate excitotoxicity is hypothesized to be an important contributor in many neurodegenerative and neuropsychiatric disorders, including Parkinson's disease, Alzheimer's disease, Huntington's disease, schizophrenia, and bipolar disorder (BD) (Mehta et al., 2013; Plitman et al., 2014). Glutamate signaling occurs through ionotropic or metabotropic glutamate receptors. The N-methyl-D-aspartate (NMDA) receptor is an ionotropic glutamate receptor that causes an influx of calcium upon activation, giving it its convulsant properties in high doses due to a dramatic increase in action potentials which can be seen as spike trains in EEG recordings (Corner et al., 2002; Ormandy et al., 1991). Activation of NMDA receptors can also result in the activation of various signaling pathways that are calcium-dependent, including activation of protein kinase C (Fukunaga et al., 1992) and release of arachidonic acid from membrane phospholipid (Dingledine et al., 1999; Lazarewicz et al., 1990; Lazarewicz et al., 1988; Weichel et al., 1999). Interestingly, chronic treatment of rats with mood stabilizers commonly used in BD, including lithium, at therapeutically relevant doses decreased alterations in brain arachidonic acid signaling produced by acute NMDA administration (Rapoport, 2014), suggesting that lithium may be able to ameliorate the effects of excitotoxicity produced by chronic hyperactivation of glutamatergic signaling. In this regard, a model for chronic excitotoxicity has been developed, involving daily i.p. injection of NMDA in rats at daily doses ranging from 25 to 130 mg/kg (Ormandy et al., 1991). The lowest dose produced brief spike trains 5–10 s duration within 10 min, which soon disappeared, suggesting that while the calcium influx produced by the dose is sufficient to produce excitation, it is not sufficient to produce convulsions (Ormandy et al., 1991). Pre-treatment with lithium did not potentiate the severity of EEG changes caused by chronic NMDA. We thought that the low dose NMDA regimen would be of interest in this study.

Excessive calcium influx caused by hyperactivation of NMDA receptors may cause oxidative stress by increasing calcium entry into the mitochondria (Clay et al., 2010; Gao et al., 2007; Giorgi et al., 2011; Hama-Tomioka et al., 2012; Lemasters et al., 2009). In turn, excessive mitochondrial calcium can disturb the electron transfer process, producing reactive oxygen species (ROS) through leakage of electrons from the electron transport chain (ETC), particularly complexes I and III. Furthermore, excess calcium can cause activation of nitric oxide synthase, resulting in increased production of nitric oxide that can form peroxynitrite, a reactive nitrogen species (RNS) (Girouard et al., 2009; Hama-Tomioka et al., 2012). RNS and ROS produced in this process can react with lipids, proteins and DNA to cause various structural and functional modifications (Barzilai and Yamamoto, 2004; Beal, 2002; Clay et al., 2010; Fariss et al., 2005). Lipid peroxidation, which can be produced by ROS, was found to correlate with white matter abnormalities in patients with BD (Versace et al., 2014). Lipid peroxidation may be a particularly relevant target to excitotoxicity produced by hyperactivation of NMDA receptors, since chronic NMDA treatment was found to increase the release of arachidonic acid, which is a favorable substrate for lipid peroxidation (Axelrod, 1990; Bosetti et al., 2002; Chang et al., 1996; Felder et al., 1990; Pellerin and Wolfe, 1991).

Therefore, the aim of this study was to examine whether excessive and chronic activation of NMDA receptors could impair functioning of the mitochondrial ETC and lipid peroxidation in the brain. First, we examined whether alterations in the mitochondrial ETC and products of lipid peroxidation occurred in the frontal cortex of rats that were given chronic (21 days) daily subconvulsive intraperitoneal (i.p.) injection of NMDA. Second, we determined if chronic lithium, at a therapeutically relevant dose that is used to treat BD, could decrease these alterations if they occurred.

2. Methods

2.1. Animals

Experiments were conducted following the “Guide for the Care and Use of Laboratory Animals” (National Institutes of Health Publication No. 86–23). Experiments were approved by the Animal Care and Use Committee of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), under Protocol 09-026. Male Fisher CDF (F-344) rats (2 months old) weighing between 200 and 250 g (Charles River Laboratories; Wilmington, MA, USA) were divided into four experimental groups. They were housed in an animal facility that had regulated temperature, humidity and a 12 h light/12 h dark cycle.

Two groups of rats were given lithium followed by NMDA treatment (Group I and II), and two groups (Group III and IV) received regular food followed by NMDA treatment. More specifically, animals in group I (Li + saline; N = 12) and II (Li + NMDA; N = 12) were fed Purina Rat Chow (Harlan, Telkad, Madison, WI, USA) ad libitum containing 1.70 g lithium chloride per kg in pelleted form for 4 weeks, followed by pellets containing 2.55 g lithium chloride per kg for an additional 2 weeks. This feeding regimen has been shown to produce plasma and brain lithium levels of 0.7 mM, which is therapeutically relevant to the treatment of BD (Bosetti et al., 2002; Chang et al., 1996). Rats in groups III (Controls; N = 12) and IV (NMDA; N = 11) received lithium-free Purina Rat Chow for 6 weeks. After 3 weeks on the diet regimen, rats in groups I and III received daily i.p. injections of 0.3 mL 0.9% NaCl for 21 days, while rats in groups II and IV received daily i.p. injections of NMDA solution (25 mg/kg body weight in 0.9% NaCl; Sigma Aldrich, St. Louis, MO, USA) for 21 days. As it has been demonstrated that NMDA mediates alterations in circadian rhythm (Shibata et al., 1994), we were sure to keep the injections at the same time to minimize the effect of any shifts in the circadian rhythm on our results. We also maintained the 12 h night/dark cycle throughout the treatment. This regimen of NMDA does not produce convulsions, but produces brief spike trains that disappear by 5 min after an injection (Ormandy et al., 1991). Three hours after the last injection, animals were sacrificed by overexposure to carbon dioxide and their brains were rapidly removed. The frontal cortex was isolated and flash frozen with methylbutane and dry ice for storage in −80 °C. The frozen samples were sent immediately by FEDEX in a dry ice container to the laboratory of Dr. Andreazza at the University of Toronto, for chemical analysis.

2.2. Mitochondrial electron transport chain complex I, III, and V

Levels of ETC complexes I, III and V were measured using the Rat/Mouse Oxidative Phosphorylation (OXPHOS) Magnetic Bead Panel (RM0XPSMAG-17K; EMD Millipore, St. Charles, Missouri, USA) with the Milliplex method following manufacturer's instructions. Briefly, samples were lysed using the lysis buffer provided in the kit and centrifuged to obtain the proteins. 5 ug of total protein was used. After the addition of the samples to the plate, antibody-containing beads were added to the samples for 2 h at room temperature, followed by the addition of detection antibodies for 1 h at room temperature. Streptavidin-phycoerythrin was added to the plate, and the samples were read on Luminex and analyzed using the xPONENT software. Results are expressed in median fluorescence intensity (MFI).

2.3. Lipid hydroperoxides

Lipid hydroperoxides were measured using the Lipid Hydroperoxide Assay Kit (705003; Cayman Chemical Company, Ann Arbor, MI, USA) according to the manufacturer's instructions. With this kit, hydroperoxides are measured using the reaction between hydroperoxides with ferrous ions to produce ferric ions, which can be detected using a chromogen. Briefly, lipid hydroperoxides in homogenized samples (100 ug total protein per sample) were extracted using chloroform and Extract R saturated methanol provided in the kit. The amount of lipid hydroperoxides was measured by adding chromogen to each sample and reading the absorbance at 500 nm using a micro-plate reader (BioTek® Instruments, Winooski, VT, USA).

2.4. 8-isoprostane

8-Isoprostane levels were measured using the 8-isoprostane EIA kit (516351; Cayman) according to the manufacturer's instructions. Using this kit, 8-isoprostane levels are measured by competitive binding between 8-isoprostane in the sample and 8-isoprostane-acetylcholinesterase provided by the kit. Levels of bound 8-isoprostane-acetylcholinesterase are measured by adding Ellman's Reagent containing the substrate to acetylcholinesterase, which is inversely proportional to 8-isoprostane in the sample. Briefly, supernatant collected from homogenizing and centrifuging the sample (50 ug total protein per sample) was incubated with 8-isoprostane-acetylcholinesterase for 8 h at 4 °C. After rinsing the wells, Ellman's Reagent was added and the samples were read at 415 nm using a micro-plate reader (BioTek® Instruments).

2.5. 4-hydroxynonenal

4-Hydroxynonenal (4-HNE) levels were measured using the ELISA method. Samples were homogenized and centrifuged to obtain total protein. 5 ug of protein was added to each well on a protein binding plate (28298-602; VWR, Ville Mont-Royal, Quebec, Canada) and incubated overnight at 4 °C. With washes between each step (5×), blocking buffer (5% BSA) was added for 1 h at room temperature (RT), followed by the primary antibody (393207, EMD Millipore) for 1 h at RT. Secondary antibody conjugated to horse radish peroxidase (7074; Cell Signaling Technology, Danvers, MA, USA) was added for 1 h at RT. TMB substrate solution was added for 20 min followed by stop solution (2M HCl). Absorbance was measured at 450 nm with 620 nm as a reference using a microplate reader (BioTek® Instruments). The absorbance was compared against that of a standard curve using known amounts of 4-HNE created by mixing 4-HNE (A8806; Sigma–Aldrich, Oakville, ON, Canada) with fatty acid free BSA (A7030; Sigma–Aldrich) as previously described (Weber et al., 2013).

2.6. Statistical analysis

SPSS v.21 was used to perform statistical analysis. Parametric distribution of data was determined using the Kolmogorov–Smirnov test. If the data were parametrically distributed, oneway ANOVA followed by Tukey's post-hoc test was used to determine between-group differences. For non-parametrically distributed data, Kruskal–Wallis test was followed by Dunn's multiple comparisons test. Correlations were assessed using Pearson's correlation test between parametric variables, and Spearman's rank correlation test if a non-parametric variable was involved. Data are represented as mean ± standard error of the mean (SEM).

3. Results

3.1. NMDA treatment decreases complex I and III

Levels (mean fluorescent intensity, MFI) of complex I, III and V were measured using the Luminex system. For complex I (Fig. 1A), there was a significant difference between the groups (F3,43 = 7.85, p < 0.01), where NMDA injections significantly decreased complex I levels compared to the control group (p < 0.01). Lithium was unable to protect against the NMDA-induced decrease in complex I, as the Li + NMDA group had significantly lower levels of complex I than the control group. For complex III, there was also a significant between-group difference (F = 7.02, p < 0.01). NMDA injections also significantly decreased complex III levels compared to the control group (p < 0.05) (Fig. 1B). Lithium prevented this NMDA-induced decrease, as the Li + NMDA group did not differ from the controls (p = 0.45). However, when the two outliers in the Li + NMDA condition were removed, Li + NMDA group significantly differed from the control group (p < 0.01). For complex V, we did not observe a between-group difference (p = 0.37) (Fig. 1C).

Fig. 1.

Fig. 1

Open in a new tab

Levels of mitochondrial electron transport chain complex I (A), III (B) and V (C). Groups are control + saline injections (ctrl + sal, N = 12), control + NMDA injections (ctrl + NMDA, N = 11), lithium + saline injections (li + sal, N = 12), and lithium + NMDA injections (li + NMDA, N = 12). Data are presented as mean fluorescence intensity (MFI), mean ± standard error of the mean. *p < 0.05, **p < 0.01.

3.2. Lipid hydroperoxides are unaffected by NMDA or lithium treatments

Concentrations (nmol/mg) of lipid hydroperoxides were not normally distributed (KS Z = 1.64, p < 0.01) (Fig. 2). Lipid hydroperoxide levels did not differ between groups (Kruskal–Wallis [K–W] = 5.04, p = 0.17).

Fig. 2.

Fig. 2

Open in a new tab

Levels of lipid hydroperoxide. Groups are control + saline injections (ctrl + sal, N = 12), control + NMDA injections (ctrl + NMDA, N = 11), lithium + saline injections (li + sal, N = 12), and lithium + NMDA injections (li + NMDA, N = 12). Data are presented as mean ± standard error of the mean.

3.3. NMDA treatment increases 8-isoprostane levels and lithium prevents this effect

Concentrations (pg/ml) of 8-isoprostane were not normally distributed (KS Z = 2.07, p < 0.01). We found a significant between-group difference (Kruskal–Wallis [K–W] = 10.94, p < 0.05). Using Dunn's multiple comparison test, animals given NMDA injections without lithium had significantly greater levels of 8-isoprostane compared to animals in the control condition (p < 0.05) and animals given lithium only (p < 0.05), suggesting that NMDA produces a significant increase in 8-isoprostane (Fig. 3). Treatment with chronic lithium prevented this NMDA-induced increase, as Li + NMDA rats did not significantly differ from the control group. 8-isoprostane levels negatively correlated with complex III levels (Spearman's rho = −0.42, p < 0.01).

Fig. 3.

Fig. 3

Open in a new tab

8-isoprostane. (A) Levels of 8-isoprostane. Groups are control + saline injections (ctrl + sal, N = 12), control + NMDA injections (ctrl + NMDA, N = 11), lithium + saline injections (li + sal, N = 12), and lithium + NMDA injections (li + NMDA, N = 12). Data are presented as mean ± standard error of the mean. *p < 0.05. (B) Correlation between 8-isoprostane levels (pg/mL) and complex III (MFI). Spearman's rho = −0.42, p < 0.01.

3.4. NMDA treatment increases 4-HNE levels and lithium prevents this effect

We found a significant between-group difference in 4-HNE levels (fmol/ug) (F3,43 = 4.08, p < 0.05) (Fig. 4). The only two groups found to significantly differ from each other were the control + NMDA and control groups (p < 0.05), suggesting that NMDA increased 4-HNE levels. Lithium diet prevented the effect of NMDA injections, such that 8-isoprostane levels in the Li + NMDA group did not differ from the control group (p = 0.11). 4-HNE levels negatively correlated with complex I levels (Pearson r = −0.45, p < 0.01).

Fig. 4.

Fig. 4

Open in a new tab

4-hydroxynonenal (4-HNE). (A) Levels of 4-HNE. Groups are control + saline injections (ctrl + sal, N = 12), control + NMDA injections (ctrl + NMDA, N = 11), lithium + saline injections (li + sal, N = 12), and lithium + NMDA injections (li + NMDA, N = 12). Data are presented as mean ± standard error of the mean. *p < 0.05. (B) Correlation between 4-HNE levels (fmol/ug) and complex I (MFI). Pearson r = −0.45, p < 0.01.

4. Discussion

The aim of this study was to examine if chronic subconvulsive NMDA injections can cause alterations in brain mitochondrial ETC components complex I, III and V, and produce lipid peroxidation in rats, and if chronic lithium treatment would modify any alterations. Studies have shown that glutamate excitotoxicity is an important feature in a number of brain diseases, including BD (Chitty et al., 2013; Gigante et al., 2012). Rats given subconvulsive injections of NMDA show increased brain arachidonic metabolism, apoptotic factors, and inflammatory cytokines (Rapoport, 2014). These levels are decreased by therapeutically relevant levels of chronically administered mood stabilizers commonly used in BD, such as lithium (Bosetti et al., 2002; Chang et al., 1996; Rapoport, 2014), suggesting that lithium may be protective against excitotoxicity.

This study is the first to demonstrate that chronic NMDA administration decreases levels of complex I and III, and increases levels of 8-isoprostane and 4-HNE, in the frontal cortex of rats. Our study also showed that chronic lithium feeding could prevent the NMDA-induced elevations in 8-isoprostane and 4-HNE. These findings suggest that excessive activation of NMDA receptors, alterations in complexes I and III, and lipid peroxidation may be closely linked.

Rats treated with NMDA had lower levels of complex I and III, but not of complex V, in the frontal cortex compared to control rats. Complexes I, III and V were chosen to be measured as hyperactivation of NMDA receptors was shown to increase oxidative stress and decrease ATP production (Mehta et al., 2013). Complexes I and III are primary producers of ROS when they are dysfunctional, and complex V is the ATP synthase (Fariss et al., 2005). Therefore, we hypothesized that they would be affected by chronic NMDA treatment. We did not measure levels of complexes II and IV, as evidence suggesting their involvement in glutamate-induced excitotoxicity is scarce, although future studies examining the effect of NMDA treatment on all components of the ETC may yield interesting results. Complex I is the first member of the ETC and is responsible for extracting electrons from NADH to transfer it to ubiquinone (Lenaz, 2001). Complex III, also known as cytochrome c reductase, acts similarly to complex I by extracting electrons from ubiquinol to reduce cytochrome c (Denis, 1986; Pramanik et al., 2011). The ability of complexes I and III to act as electron transporters suggests that reduced levels of these complexes may increase generation of oxidative stress from the mitochondria by allowing electrons to leak from the system to react with molecular oxygen, generating ROS (Janssen et al., 2006; Li et al., 2003; Shrotriya et al., 2015). Oxidative damage can disrupt enzymes and transporters, change expression of certain genes or activate inflammation through redox sensors and activate apoptosis (Fariss et al., 2005; Kim et al., 2014; Naoi et al., 2005; Ray et al., 2012; Zhou et al., 2011).

These findings also suggest that increased glutamatergic activity may contribute to decreased levels of complexes I and III. One possible cause is protein misfolding from excess ROS and RNS produced by over-activation of NMDA receptors (Gu et al., 2010). Furthermore, excitotoxicity produced by increased glutamatergic signaling can cause mitochondrial fission and fragmentation, which could also contribute to lower levels of complexes I and III (Gu et al., 2010). A previous study reported that glutamate decreased the activity of complexes I and III through the NMDA receptor (Rego et al., 2000), in agreement with our findings. The exact mechanism by which NMDA decreases complex I and III levels needs further exploration.

Lithium, which is the most commonly prescribed mood stabilizer in BD, was shown to prevent the NMDA-induced reduction in complex III. However, this effect was no longer significant after 2 outliers were removed from the lithium + NMDA group. Lithium also did not change levels of complex I. Previous studies have shown that lithium pre-treatment protects against rotenone-induced reduction in complex I activity (Nascimento et al., 2015) and increases activity levels of complexes I and III (Maurer et al., 2009), while this is the first time lithium's effect on complex I and III levels was explored. This suggests that lithium's protective effects shown in vitro in previous studies may not be due to its effects on the expression levels of complexes I and III.

In agreement with our findings on levels of complex I and III, brain levels of 8-isoprostane and 4-HNE were increased in animals given NMDA. Lipid peroxidation is one of the most consistent findings in patients with BD, as shown by two meta-analyses examining oxidative stress in BD (Andreazza et al., 2008; Brown et al., 2014). These new findings suggest that hyperactivation of the glutamate system may contribute to lipid peroxidation in BD. Formation of 8-isoprostane, an end product of lipid peroxidation, occurs when a free-radical oxidizes arachidonic acid through an enzyme-independent reaction (Montuschi et al., 2004). Previous studies demonstrated that chronic NMDA treatment at subconvulsive doses increased the release of arachidonic acid and the activity of calcium-dependent cytosolic phospholipase A2, which releases arachidonic acid from phospholipids (Axelrod, 1990; Bosetti et al., 2002; Chang et al., 1996; Felder et al., 1990; Pellerin and Wolfe, 1991). Hence, NMDA treatment may have increased 8-isoprostane by increasing arachidonic acid release. Importantly, 8-isoprostane levels were found to be increased in pre-frontal cortex of patients with BD (Andreazza et al., 2013), suggesting that excessive glutamate signaling may contribute to its increased production.

4-HNE is an aldehyde product of lipid peroxidation that is highly toxic (Niki, 2009). 4-HNE can react with thiol groups in proteins and peptides to form adducts (Dalleau et al., 2013; Doorn and Petersen, 2003; Niki, 2009; Pizzimenti et al., 2013) and cause apoptosis or necrosis; it induces apoptosis at lower concentrations and necrosis at higher levels (Haupt et al., 2003). Patients with BD were reported to have greater levels of 4-HNE in the anterior cingulate cortex (Wang et al., 2009) and frontal cortex (Andreazza et al., 2013), suggesting that excess glutamate signaling also may contribute to higher levels of 4-HNE in patients with BD. While NMDA administration to rats increased brain levels of 4-HNE and 8-isoprostane, it did not significantly change lipid hydroperoxide levels. As lipid hydroperoxides are primary products of lipid peroxidation (Niki, 2009), this suggests that the majority of the primary products were further oxidized to produce end products, such as 8-isoprostane or 4-HNE. These findings strongly suggest that NMDA treatment increases oxidative stress in the frontal cortex. Increased activation of NMDA receptors can cause excessive influx of calcium into the cytoplasm, resulting in mitochondrial damage that can lead to increased production of ROS from the mitochondria (Gao et al., 2007). Activation of NMDA receptors can also increase the activity of NADPH oxidase, which releases superoxide anions to produce oxidative stress (Brennan et al., 2009; Girouard et al., 2009). In addition, our findings suggest that hyperactivation of NMDA receptors may decrease complex I and III, which could result in greater production of ROS. Interestingly, 4-HNE levels were negatively correlated with complex I, and 8-isoprostane levels were negatively correlated with complex III, suggesting that complex I and III levels may be linked to lipid peroxidation. Examining whether different oxidative modifications are linked to different abnormalities of the mitochondrial ETC is beyond the scope of this study, but these findings are in agreement with previous studies suggesting that oxidative modifications are specific processes that depend on multiple factors (Andreazza et al., 2013; Nascimento et al., 2015).

Chronic lithium prevented the NMDA-induced increase in 4-HNE and 8-isoprostane levels, which is in agreement with previous studies demonstrating lithium's antioxidant effects (Cui et al., 2007; Jornada et al., 2011; Machado-Vieira et al., 2007; Shao et al., 2008). Indeed, lithium was able to prevent protein oxidation, nitration and lipid peroxidation produced by rotenone and amphetamine in cell models and in animals (Jornada et al., 2011; Nascimento et al., 2015; Tan et al., 2011). Lithium was shown to reduce oxidative stress through multiple different mechanisms, including increasing components of the glutathione system, which is the most important antioxidant in the brain (Cui et al., 2007; Nascimento et al., 2015; Shao et al., 2008). Lithium may have exerted its effects by directly reducing glutamate signaling (Malhi et al., 2013). Indeed, studies have shown that lithium causes downregulation of NMDA receptors, increases glutamate re-uptake, and prevents excitotoxic processes induced by hyperactivation of NMDA receptors by inhibiting calcium entry (for review, see Malhi et al., 2013 (Malhi et al., 2013). While lithium was shown to protect against the NMDA-induced decrease in complex III levels, as this effect was no longer significant after removing outliers, lithium may act through different pathways to ameliorate NMDA-induced lipid peroxidation. Lithium was also shown to prevent NMDA-induced increase in phospholipase A2 and arachidonic acid turnover in previous studies (Bosetti et al., 2002; Chang et al., 1996), suggesting that lithium prevented NMDA-induced increase in 8-isoprostane by reducing the release and subsequent metabolism of arachidonic acid.

The findings of this study must be interpreted in light of its limitations. First, while we measured levels of complexes I and III, we did not measure their activities. Examining the effect of chronic NMDA treatment on the activity of these complexes may yield interesting results. Furthermore, while we decided to focus on lipid peroxidation in this study due to its high relevance to BD and its link to NMDA receptors through the arachidonic acid cascade, it may be useful to also examine other products of oxidative stress, such as carbonylation and nitration, to better understand the relation between glutamatergic hyperactivity and oxidative stress. Finally, examining the relation between alterations in peripheral systems and the brain caused by NMDA treatment may increase our understanding of the influence of glutamatergic hyperactivity on mitochondrial dysfunction and oxidative stress.

In conclusion, our findings demonstrate that chronic administration of subconvulsive doses of i.p. NMDA in rats decreased levels of mitochondrial ETC complexes I and III, and increased lipid peroxidation products, 8-isoprostane and 4-HNE, in the frontal cortex. Chronic treatment at a clinically relevant dose with lithium prevented the NMDA-induced effects on 8-isoprostane and 4-HNE. These findings suggest that glutamate signaling, mitochondrial functioning and lipid peroxidation are closely linked, such that excessive activation of NMDA receptors may cause dysfunction in the mitochondrial ETC and increase levels of lipid peroxidation. Lithium exerted partial protection by decreasing lipid peroxidation. As glutamatergic hyperactivity plays a role in several brain disorders, including BD (Clinton and Meador-Woodruff, 2004; Gigante et al., 2012), these findings also suggest that decreasing glutamatergic signaling, or blocking pathways that occur downstream of NMDA receptor activation may be beneficial for the treatment of these diseases. Support for this for BD comes from recent clinical trials with anti-glutamatergic drugs (Machado-Vieira et al., 2012).

Acknowledgments

We thank Ms. Mei Chen for preparing and injecting the rats, and Dr. Jagadeesh Rao for dissecting the brains. The authors acknowledge CIHR and Brain & Behavior Research Foundation as sources of funding. Helena K. Kim was supported by the Ontario Mental Health Foundation Scholarship.

Funding sources

The work by SIR was entirely supported by the Intramural Program of the National Institute on Aging, NIH. ACA receives grant funding from Canadian Institute of Health Research (CIHR MOP-133439) and Brain and Behavior Research Foundation (NARSAD). HKK was supported by the Ontario Mental Health Foundation. Funding sources had no involvement in study design, collection, analysis and interpretation of data, and preparation of the manuscript.

Footnotes

Contributors

Helena Kyunghee Kim: HKK contributed to designing the study, performing the experiments, statistical analysis, preparation and submission of the manuscript.

Cameron Isaacs-Trepanier: CIT contributed to performing the experiments, statistical analysis, and preparation of the manuscript.

Nika Elmi: NE contributed to performing the experiments, statistical analysis, and preparation of the manuscript.

Stanley I. Rapoport: SIR contributed to designing the study, treating the animals, preparing the brain tissue for shipment, preparation and submission of the manuscript.

Ana Cristina Andreazza: ACA contributed to designing the study, statistical analysis, preparation and submission of the manuscript.

*All authors have approved the final article.

Conflicts of interest

Authors have no conflict of interest to disclose.

References

  1. Andreazza AC, Kauer-Sant'anna M, Frey BN, Bond DJ, Kapczinski F, Young LT, Yatham LN. Oxidative stress markers in bipolar disorder: a meta-analysis. J. Affect Disord. 2008;111:135–144. doi: 10.1016/j.jad.2008.04.013. [DOI] [PubMed] [Google Scholar]
  2. Andreazza AC, Wang JF, Salmasi F, Shao L, Young LT. Specific subcellular changes in oxidative stress in prefrontal cortex from patients with bipolar disorder. J. Neurochem. 2013 Nov;127(4):552–561. doi: 10.1111/jnc.12316. http://dx.doi.org/10.1111/jnc.12316. Epub 2013 Jun 11. [DOI] [PubMed] [Google Scholar]
  3. Axelrod J. Receptor-mediated activation of phospholipase A2 and arachidonic acid release in signal transduction. Biochem. Soc. Trans. 1990;18:503–507. doi: 10.1042/bst0180503. [DOI] [PubMed] [Google Scholar]
  4. Barzilai A, Yamamoto K. DNA damage responses to oxidative stress. DNA Repair (Amst) 2004;3:1109–1115. doi: 10.1016/j.dnarep.2004.03.002. [DOI] [PubMed] [Google Scholar]
  5. Beal MF. Oxidatively modified proteins in aging and disease. Free Radic. Biol. Med. 2002;32:797–803. doi: 10.1016/s0891-5849(02)00780-3. [DOI] [PubMed] [Google Scholar]
  6. Bosetti F, Rintala J, Seemann R, Rosenberger TA, Contreras MA, Rapoport SI, Chang MC. Chronic lithium downregulates cyclooxygenase-2 activity and prostaglandin E(2) concentration in rat brain. Mol. Psychiatry. 2002;7:845–850. doi: 10.1038/sj.mp.4001111. [DOI] [PubMed] [Google Scholar]
  7. Brennan AM, Suh SW, Won SJ, Narasimhan P, Kauppinen TM, Lee H, Edling Y, Chan PH, Swanson RA. NADPH oxidase is the primary source of superoxide induced by NMDA receptor activation. Nat. Neurosci. 2009;12:857–863. doi: 10.1038/nn.2334. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brown NC, Andreazza AC, Young LT. An updated meta-analysis of oxidative stress markers in bipolar disorder. Psychiatry Res. 2014;218:61–68. doi: 10.1016/j.psychres.2014.04.005. [DOI] [PubMed] [Google Scholar]
  9. Chang MC, Grange E, Rabin O, Bell JM, Allen DD, Rapoport SI. Lithium decreases turnover of arachidonate in several brain phospholipids. Neurosci. Lett. 1996;220:171–174. doi: 10.1016/s0304-3940(96)13264-x. [DOI] [PubMed] [Google Scholar]
  10. Chitty KM, Lagopoulos J, Lee RS, Hickie IB, Hermens DF. A systematic review and meta-analysis of proton magnetic resonance spectroscopy and mismatch negativity in bipolar disorder. Eur. Neuropsychopharmacol. 2013;23:1348–1363. doi: 10.1016/j.euroneuro.2013.07.007. [DOI] [PubMed] [Google Scholar]
  11. Clay HB, Sillivan S, Konradi C. Mitochondrial dysfunction and pathology in bipolar disorder and schizophrenia. Int. J. Dev. Neurosci. 2010;29:311–324. doi: 10.1016/j.ijdevneu.2010.08.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Clinton SM, Meador-Woodruff JH. Thalamic dysfunction in schizophrenia: neurochemical, neuropathological, and in vivo imaging abnormalities. Schizophr. Res. 2004;69:237–253. doi: 10.1016/j.schres.2003.09.017. [DOI] [PubMed] [Google Scholar]
  13. Corner MA, van Pelt J, Wolters PS, Baker RE, Nuytinck RH. Physiological effects of sustained blockade of excitatory synaptic transmission on spontaneously active developing neuronal networks–an inquiry into the reciprocal linkage between intrinsic biorhythms and neuroplasticity in early ontogeny. Neurosci. Biobehav Rev. 2002;26:127–185. doi: 10.1016/s0149-7634(01)00062-8. [DOI] [PubMed] [Google Scholar]
  14. Cui J, Shao L, Young LT, Wang JF. Role of glutathione in neuroprotective effects of mood stabilizing drugs lithium and valproate. Neuroscience. 2007;144:1447–1453. doi: 10.1016/j.neuroscience.2006.11.010. [DOI] [PubMed] [Google Scholar]
  15. Dalleau S, Baradat M, Guéraud F, Huc L. Cell death and diseases related to oxidative stress: 4-hydroxynonenal (HNE) in the balance. Cell Death Differ. 2013;20:1615–1630. doi: 10.1038/cdd.2013.138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Denis M. Structure and function of cytochrome-c oxidase. Biochimie. 1986;68:459–470. doi: 10.1016/s0300-9084(86)80013-x. [DOI] [PubMed] [Google Scholar]
  17. Dingledine R, Borges K, Bowie D, Traynelis SF. The glutamate receptor ion channels. Pharmacol. Rev. 1999;51:7–61. [PubMed] [Google Scholar]
  18. Doorn JA, Petersen DR. Covalent adduction of nucleophilic amino acids by 4-hydroxynonenal and 4-oxononenal. Chem. Biol. Interact. 2003;143–144:93–100. doi: 10.1016/s0009-2797(02)00178-3. [DOI] [PubMed] [Google Scholar]
  19. Fariss MW, Chan CB, Patel M, Van Houten B, Orrenius S. Role of mitochondria in toxic oxidative stress. Mol. Interv. 2005;5:94–111. doi: 10.1124/mi.5.2.7. [DOI] [PubMed] [Google Scholar]
  20. Felder CC, Kanterman RY, Ma AL, Axelrod J. Serotonin stimulates phospholipase A2 and the release of arachidonic acid in hippocampal neurons by a type 2 serotonin receptor that is independent of inositolphospholipid hydrolysis. Proc. Natl. Acad. Sci. U. S. A. 1990;87:2187–2191. doi: 10.1073/pnas.87.6.2187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fukunaga K, Soderling TR, Miyamoto E. Activation of Ca2+/calmodulin-dependent protein kinase II and protein kinase C by glutamate in cultured rat hippocampal neurons. J. Biol. Chem. 1992;267:22527–22533. [PubMed] [Google Scholar]
  22. Gao X, Xu X, Pang J, Zhang C, Ding JM, Peng X, Liu Y, Cao JM. NMDA receptor activation induces mitochondrial dysfunction, oxidative stress and apoptosis in cultured neonatal rat cardiomyocytes. Physiol. Res. 2007;56:559–569. doi: 10.33549/physiolres.931053. [DOI] [PubMed] [Google Scholar]
  23. Gigante AD, Bond DJ, Lafer B, Lam RW, Young LT, Yatham LN. Brain glutamate levels measured by magnetic resonance spectroscopy in patients with bipolar disorder: a meta-analysis. Bipolar Disord. 2012;14:478–487. doi: 10.1111/j.1399-5618.2012.01033.x. [DOI] [PubMed] [Google Scholar]
  24. Giorgi C, Agnoletto C, Bononi A, Bonora M, De Marchi E, Marchi S, Missiroli S, Patergnani S, Poletti F, Rimessi A, Suski JM, Wieckowski MR, Pinton P. Mitochondrial calcium homeostasis as potential target for mitochondrial medicine. Mitochondrion. 2011;12:77–85. doi: 10.1016/j.mito.2011.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Girouard H, Wang G, Gallo EF, Anrather J, Zhou P, Pickel VM, Iadecola C. NMDA receptor activation increases free radical production through nitric oxide and NOX2. J. Neurosci. 2009;29:2545–2552. doi: 10.1523/JNEUROSCI.0133-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Gu Z, Nakamura T, Lipton SA. Redox reactions induced by nitrosative stress mediate protein misfolding and mitochondrial dysfunction in neurodegenerative diseases. Mol. Neurobiol. 2010;41:55–72. doi: 10.1007/s12035-010-8113-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hama-Tomioka K, Kinoshita H, Nakahata K, Kondo T, Azma T, Kawahito S, Hatakeyama N, Matsuda N. Roles of neuronal nitric oxide synthase, oxidative stress, and propofol in N-methyl-D-aspartate-induced dilatation of cerebral arterioles. Br. J. Anaesth. 2012;108:21–29. doi: 10.1093/bja/aer368. [DOI] [PubMed] [Google Scholar]
  28. Haupt S, Berger M, Goldberg Z, Haupt Y. Apoptosis - the p53 network. J. Cell Sci. 2003;116:4077–4085. doi: 10.1242/jcs.00739. [DOI] [PubMed] [Google Scholar]
  29. Janssen RJ, Nijtmans LG, van den Heuvel LP, Smeitink JA. Mitochondrial complex I: structure, function and pathology. J. Inherit. Metab. Dis. 2006;29:499–515. doi: 10.1007/s10545-006-0362-4. [DOI] [PubMed] [Google Scholar]
  30. Jornada LK, Valvassori SS, Steckert AV, Moretti M, Mina F, Ferreira CL, Arent CO, Dal-Pizzol F, Quevedo J. Lithium and valproate modulate antioxidant enzymes and prevent ouabain-induced oxidative damage in an animal model of mania. J. Psychiatr. Res. 2011;45:162–168. doi: 10.1016/j.jpsychires.2010.05.011. [DOI] [PubMed] [Google Scholar]
  31. Kim HK, Andreazza AC, Yeung PY, Isaacs-Trepanier C, Young LT. Oxidation and nitration in dopaminergic areas of the prefrontal cortex from patients with bipolar disorder and schizophrenia. J. Psychiatry Neurosci. 2014;39:276–285. doi: 10.1503/jpn.130155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lazarewicz JW, Wroblewski JT, Costa E. N-methyl-D-aspartate-sensitive glutamate receptors induce calcium-mediated arachidonic acid release in primary cultures of cerebellar granule cells. J. Neurochem. 1990;55:1875–1881. doi: 10.1111/j.1471-4159.1990.tb05771.x. [DOI] [PubMed] [Google Scholar]
  33. Lazarewicz JW, Wroblewski JT, Palmer ME, Costa E. Activation of N-methyl-D-aspartate-sensitive glutamate receptors stimulates arachidonic acid release in primary cultures of cerebellar granule cells. Neuropharmacology. 1988;27:765–769. doi: 10.1016/0028-3908(88)90088-3. [DOI] [PubMed] [Google Scholar]
  34. Lemasters JJ, Theruvath TP, Zhong Z, Nieminen AL. Mitochondrial calcium and the permeability transition in cell death. Biochim. Biophys. Acta. 2009;1787:1395–1401. doi: 10.1016/j.bbabio.2009.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Lenaz G. The mitochondrial production of reactive oxygen species: mechanisms and implications in human pathology. IUBMB Life. 2001;52:159–164. doi: 10.1080/15216540152845957. [DOI] [PubMed] [Google Scholar]
  36. Li N, Ragheb K, Lawler G, Sturgis J, Rajwa B, Melendez JA, Robinson JP. Mitochondrial complex I inhibitor rotenone induces apoptosis through enhancing mitochondrial reactive oxygen species production. J. Biol. Chem. 2003;278:8516–8525. doi: 10.1074/jbc.M210432200. [DOI] [PubMed] [Google Scholar]
  37. Machado-Vieira R, Andreazza AC, Viale CI, Zanatto V, Cereser V, Jr, da Silva Vargas R, Kapczinski F, Portela LV, Souza DO, Salvador M, Gentil V. Oxidative stress parameters in unmedicated and treated bipolar subjects during initial manic episode: a possible role for lithium antioxidant effects. Neurosci. Lett. 2007;421:33–36. doi: 10.1016/j.neulet.2007.05.016. [DOI] [PubMed] [Google Scholar]
  38. Machado-Vieira R, Ibrahim L, Henter ID, Zarate CA. Novel glutamatergic agents for major depressive disorder and bipolar disorder. Pharmacol. Biochem. Behav. 2012;100:678–687. doi: 10.1016/j.pbb.2011.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Malhi GS, Tanious M, Das P, Coulston CM, Berk M. Potential mechanisms of action of lithium in bipolar disorder. Current understanding. CNS Drugs. 2013;27:135–153. doi: 10.1007/s40263-013-0039-0. [DOI] [PubMed] [Google Scholar]
  40. Maurer IC, Schippel P, Volz HP. Lithium-induced enhancement of mitochondrial oxidative phosphorylation in human brain tissue. Bipolar Disord. 2009;11:515–522. doi: 10.1111/j.1399-5618.2009.00729.x. [DOI] [PubMed] [Google Scholar]
  41. Mehta A, Prabhakar M, Kumar P, Deshmukh R, Sharma PL. Excitotoxicity: bridge to various triggers in neurodegenerative disorders. Eur. J. Pharmacol. 2013;698:6–18. doi: 10.1016/j.ejphar.2012.10.032. [DOI] [PubMed] [Google Scholar]
  42. Montuschi P, Barnes PJ, Roberts LJ. Isoprostanes: markers and mediators of oxidative stress. FASEB J. 2004;18:1791–1800. doi: 10.1096/fj.04-2330rev. [DOI] [PubMed] [Google Scholar]
  43. Naoi M, Maruyama W, Shamoto-Nagai M, Yi H, Akao Y, Tanaka M. Oxidative stress in mitochondria: decision to survival and death of neurons in neurodegenerative disorders. Mol. Neurobiol. 2005;31:81–93. doi: 10.1385/MN:31:1-3:081. [DOI] [PubMed] [Google Scholar]
  44. Nascimento C, Kim HK, Young LT, Mendonça KM, Grinberg LT, Lafer B, Andreazza AC. Glutathione-mediated effects of lithium in decreasing protein oxidation induced by mitochondrial complex I dysfunction. J. Neural Transm. (Vienna) 2015 Jun;122(6):741–746. doi: 10.1007/s00702-014-1318-8. http://dx.doi.org/10.1007/s00702-014-1318-8. Epub 2014 Sep 27. [DOI] [PubMed] [Google Scholar]
  45. Niki E. Lipid peroxidation: physiological levels and dual biological effects. Free Radic. Biol. Med. 2009;47:469–484. doi: 10.1016/j.freeradbiomed.2009.05.032. [DOI] [PubMed] [Google Scholar]
  46. Ormandy GC, Song L, Jope RS. Analysis of the convulsant-potentiating effects of lithium in rats. Exp. Neurol. 1991;111:356–361. doi: 10.1016/0014-4886(91)90103-j. [DOI] [PubMed] [Google Scholar]
  47. Pellerin L, Wolfe LS. Release of arachidonic acid by NMDA-receptor activation in the rat hippocampus. Neurochem. Res. 1991;16:983–989. doi: 10.1007/BF00965841. [DOI] [PubMed] [Google Scholar]
  48. Pizzimenti S, Ciamporcero E, Daga M, Pettazzoni P, Arcaro A, Cetrangolo G, Minelli R, Dianzani C, Lepore A, Gentile F, Barrera G. Interaction of aldehydes derived from lipid peroxidation and membrane proteins. Front. Physiol. 2013;4:242. doi: 10.3389/fphys.2013.00242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Plitman E, Nakajima S, de la Fuente-Sandoval C, Gerretsen P, Chakravarty MM, Kobylianskii J, Chung JK, Caravaggio F, Iwata Y, Remington G, Graff-Guerrero A. Glutamate-mediated excitotoxicity in schizophrenia: a review. Eur. Neuropsychopharmacol. 2014;24:1591–1605. doi: 10.1016/j.euroneuro.2014.07.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Pramanik KC, Boreddy SR, Srivastava SK. Role of mitochondrial electron transport chain complexes in capsaicin mediated oxidative stress leading to apoptosis in pancreatic cancer cells. PLoS One. 2011;6:e20151. doi: 10.1371/journal.pone.0020151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Rapoport SI. Lithium and the other mood stabilizers effective in bipolar disorder target the rat brain arachidonic acid cascade. ACS Chem. Neurosci. 2014;5:459–467. doi: 10.1021/cn500058v. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Ray PD, Huang BW, Tsuji Y. Reactive oxygen species (ROS) homeostasis and redox regulation in cellular signaling. Cell Signal. 2012;24:981–990. doi: 10.1016/j.cellsig.2012.01.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Rego AC, Santos MS, Oliveira CR. Glutamate-mediated inhibition of oxidative phosphorylation in cultured retinal cells. Neurochem. Int. 2000;36:159–166. doi: 10.1016/s0197-0186(99)00107-2. [DOI] [PubMed] [Google Scholar]
  54. Shao L, Cui J, Young LT, Wang JF. The effect of mood stabilizer lithium on expression and activity of glutathione s-transferase isoenzymes. Neuroscience. 2008;151:518–524. doi: 10.1016/j.neuroscience.2007.10.041. [DOI] [PubMed] [Google Scholar]
  55. Shibata S, Watanabe A, Hamada T, Ono M, Watanabe S. N-methyl-D-aspartate induces phase shifts in circadian rhythm of neuronal activity of rat SCN in vitro. Am. J. Physiol. 1994;267:R360–R364. doi: 10.1152/ajpregu.1994.267.2.R360. [DOI] [PubMed] [Google Scholar]
  56. Shrotriya S, Deep G, Lopert P, Patel M, Agarwal R, Agarwal C. Grape seed extract targets mitochondrial electron transport chain complex III and induces oxidative and metabolic stress leading to cytoprotective autophagy and apoptotic death in human head and neck cancer cells. Mol. Carcinog. 2015 Dec;54(12):1734–1747. doi: 10.1002/mc.22246. http://dx.doi.org/10.1002/mc.22246. Epub 2014 Dec 31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Tan H, Young L, Shao L, Che Y, Honer W, Wang J. Mood stabilizer lithium inhibits amphetamine-increased 4-hydroxynonenal-protein adducts in rat frontal cortex. Int J Neuropsychopharmacol. 2011:1–11. doi: 10.1017/S1461145711001416. [DOI] [PubMed] [Google Scholar]
  58. Versace A, Andreazza AC, Young LT, Fournier JC, Almeida JR, Stiffler RS, Lockovich JC, Aslam HA, Pollock MH, Park H, Nimgaonkar VL, Kupfer DJ, Phillips ML. Elevated serum measures of lipid peroxidation and abnormal prefrontal white matter in euthymic bipolar adults: toward peripheral biomarkers of bipolar disorder. Mol. Psychiatry. 2014 Feb;19(2):200–208. doi: 10.1038/mp.2012.188. http://dx.doi.org/10.1038/mp.2012.188. Epub 2013 Jan 29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Wang JF, Shao L, Sun X, Young LT. Increased oxidative stress in the anterior cingulate cortex of subjects with bipolar disorder and schizophrenia. Bipolar Disord. 2009;11:523–529. doi: 10.1111/j.1399-5618.2009.00717.x. [DOI] [PubMed] [Google Scholar]
  60. Weber D, Milkovic L, Bennett SJ, Griffiths HR, Zarkovic N, Grune T. Measurement of HNE-protein adducts in human plasma and serum by ELISA-Comparison of two primary antibodies. Redox Biol. 2013;1:226–233. doi: 10.1016/j.redox.2013.01.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Weichel O, Hilgert M, Chatterjee SS, Lehr M, Klein J. Bilobalide, a constituent of Ginkgo biloba, inhibits NMDA-induced phospholipase A2 activation and phospholipid breakdown in rat hippocampus. Naunyn Schmiedeb. Arch. Pharmacol. 1999;360:609–615. doi: 10.1007/s002109900131. [DOI] [PubMed] [Google Scholar]
  62. Zhou R, Yazdi AS, Menu P, Tschopp J. A role for mitochondria in NLRP3 inflammasome activation. Nature. 2011;469:221–225. doi: 10.1038/nature09663. [DOI] [PubMed] [Google Scholar]

RESOURCES