Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Apr 9.
Published in final edited form as: J Psychiatr Res. 2013 Jun 18;47(9):1247–1253. doi: 10.1016/j.jpsychires.2013.05.001

Cerebrospinal fluid glutamate concentration correlates with impulsive aggression in human subjects

Emil F Coccaro a,b,*, Royce Lee a,b, Paul Vezina b
PMCID: PMC3980459  NIHMSID: NIHMS568685  PMID: 23791397

Abstract

Neurochemical studies have pointed to a modulatory role in human aggression for various central neurotransmitters. Some (e.g., serotonin) appear to play an inhibitory role, while others appear to play a facilitator role. While recent animal studies of glutaminergic activity suggest a facilitator role for central glutamate in the modulation of aggression, no human studies of central glutaminergic indices have yet been reported regarding aggression. Basal lumbar cerebrospinal fluid (CSF) was obtained from 38 physically healthy subjects with DSM-IV Personality Disorder (PD: n = 28) and from Healthy Volunteers (HV: n = 10) and assayed for glutamate, and other neurotransmitters, in CSF and correlated with measures of aggression and impulsivity. CSF Glutamate levels did not differ between the PD and HC subjects but did directly correlate with composite measures of both aggression and impulsivity and a composite measure of impulsive aggression in both groups. These data suggest a positive relationship between CSF Glutamate levels and measures of impulsive aggression in human subjects. Thus, glutamate function may contribute to the complex central neuromodulation of impulsive aggression in human subjects.

Keywords: CSF, Glutamate, Aggression

1. Introduction

Glutamate is the most abundant excitatory neurotransmitter in the vertebrate nervous system (Niciu et al., 2012). Glutamate is stored in vesicles at chemical synapses where nerve impulses trigger release of glutamate from the pre-synaptic neuron onto post-synaptic glutamate receptors such as the ionotropic NMDA and AMPA/Kainate receptors and the G-protein coupled metabotropic glutamate receptors. Glutamate plays an important role in brain synaptic plasticity and is involved in a number of cognitive functions including learning and memory in the hippocampus, neocortex, and other brain regions.

Glutamate has been implicated to play a role in a variety of neuropsychiatric disorders including schizophrenia (Lin et al., 2012), mood disorder (Machado-Vieira et al., 2012), anxiety disorders (Riaza Bermudo-Soriano et al., 2012), addictive disorders (Olive et al., 2012), and other neuropsychiatric disorders (e.g., Hu et al., 2012; Carlson, 2012). While little work has been reported on the role of glutamate in human aggressive behavior, preclinical studies suggest that stimulation of central glutamate receptors typically increases aggressive behavior in lower mammals.

As demonstrated in a number of preclinical studies in rodents, an excitatory amino acid pathway from the medial hypothalamus (MH) to the periaqueductal gray (PAG) is associated with aggressive behavior (Beart et al., 1998, 1990; Beitz, 1989). In the rat, there is a dense and distinct group of glutamatergic neurons expressing glutamate transporter protein over the entire hypothalamic attack area, with the rostral portion predominantly containing glutamatergic, and the caudal portion having both glutaminergic and, to a lesser degree, GABAergic, neurons (Hrabovszky et al., 2005). Microinjections of glutamate into the cat PAG elicit defensive rage (Bandler, 1984), a finding consistent with the release of glutamate by MH neurons and the activation of PAG neurons in the expression of defensive rage in the cat. This was confirmed in subsequent studies demonstrating that pretreatment with the glutamate antagonist kynurenic acid blocked MH facilitation of PAG elicited defensive rage, and that NMDA injected into PAG defensive rage sites facilitated the rage response elicited from that site (Lu et al., 1992). Administration of an NMDA receptor antagonist into the PAG blocked MH facilitation of PAG-elicited defensive rage. The antagonist dose-dependently suppressed defensive rage elicited by stimulation of the MH (Schubert et al., 1996). This study also reported that a considerable number of glutamate neurons within the anteromedial hypothalamus project to PAG defensive rage sites (Schubert et al., 1996).

Excitatory inputs from the basal amygdala also project to the PAG and there is evidence for PAG NMDA receptor mediated defensive rage following their stimulation (Shaikh et al., 1994); basal amygdaloid neurons projecting to PAG defensive rage sites also stain immunopositive for glutamate. In addition, mice bred for reduction of function in the NMDA R1 subunit display an absence of species-typical fighting in the resident intruder model of aggression (Duncan et al., 2004).

Preclinical studies of the other ionotropic glutamate receptor, AMPA, and as well as metabotropic glutamate receptors support a glutamate hypothesis of aggressive behavior. For example, mice deficient for the AMPA receptor GluR-A1 subunit are less aggressive than their wild-type counterparts (Vekovischeva et al., 2004) and treatment with AMPA receptor antagonists also reduces aggression in aggressive mice strains (Vekovischeva et al., 2007). Knockout of the GluA3-AMPA receptor subunit in mice is associated with a reduction in aggressive behavior (Adamczyk et al., 2012). In addition, genome-wide scans to identify aggression quantitative trait loci in aggressive mice strains find that the Gria3 gene, which encodes for a subunit of the AMPA3 receptor, accounts for the strain differences in aggressive behavior in the resident-intruder mouse model of aggression (Brodkin et al., 2002). Finally, selective mGlu-1 (Navarro et al., 2008) and mGlu-5 (Navarro et al., 2006), receptor blockade reduce aggression in mice models of aggression (Navarro et al., 2006). In contrast, agonist stimulation of auto-inhibitory mGlu-2/3 (Ago et al., 2012) or mGlu-7 (Navarro et al., 2009) receptors reduces aggression in mice.

Given the results of these various preclinical studies, we sought to explore if cerebrospinal fluid (CSF) Glutamate would be associated with aggression and/or impulsivity in personality disordered and healthy volunteer subjects. We hypothesized that CSF Glutamate would correlate directly with measures of aggression and/or impulsivity.

2. Methods

2.1. Subjects

Thirty-eight physically healthy subjects participated in this study. All subjects were medically healthy and were systematically evaluated in regard to aggressive and other behaviors as part of a larger program designed to study the biological correlates of impulsive aggressive and other personality-related behaviors. Subjects were recruited through public service announcements seeking out individuals who considered themselves to have difficulty managing their aggressive behaviors and, non-aggressive individuals interested and willing to participate in biological studies of personality traits. Subjects with a life history of bipolar disorder, schizophrenia (or other psychotic disorder), mental retardation, or current substance dependence disorder, were excluded from this study. Medical health of all subjects was documented by a comprehensive medical history and physical examination and included a drug screen for amphetamine, barbiturates, benzodiazepines, cocaine, opiates, methadone, methamphetamine, phencyclidine, oxycodone, and marijuana (no one who tested positive for any substance was entered into the study). All subjects gave informed consent and signed the informed consent document approved by our Committee for the Protection of Human Subjects (IRB).

2.2. Diagnostic assessment

Axis I and Axis II Personality Disorder diagnoses were made according to DSM-IV criteria (APA, 1994). The diagnosis of Intermittent Explosive Disorder was made by Integrated Research Criteria as previously described (IED-IR: Coccaro, 2011, 2012). Diagnoses were assessed and assigned through a best estimate process as described in previous reports (Bunce et al., 2005).

Twenty-eight subjects met DSM-IV criteria for a Personality Disorder (PD) and ten subjects had no evidence of any DSM-IV Axis I or II psychopathology (Healthy Volunteers: HV). Eighteen of the PD subjects met DSM-IV criteria for a specific personality disorder as follows: a) Cluster A (n = 8), i.e., Paranoid (n = 6), Schizoid (n = 3), Schizotypal (n = 1); b) Cluster B (n = 10), i.e., Borderline (n = 5), Antisocial (n = 3); Narcissistic (n = 2); Histrionic (n = 3); c) Cluster C (n = 7), i.e., Obsessive-Compulsive (n = 5), Avoidant (n = 1); Dependent (n = 1). The remaining ten subjects were diagnosed as Personality Disorder-Not Otherwise Specified (PD-NOS). These subjects met DSM-IV general criteria for personality disorder, had pathological personality traits from a variety of personality disorder categories and had clear evidence of impaired psychosocial functioning (mean GAF score = 63.4 ± 7.5). Just over half of PD subjects had a life history of at least one Axis I disorder (15 of 28) and three-quarters had a current history of at least one Axis I disorder (21 of 28). Current Axis I disorders were as follows: Any Mood Disorder (n = 5): major depression (n = 1), dysthymia (n = 3), depressive disorder-NOS (n = 1); Any Anxiety Disorder (n = 3), i.e., all phobic (n = 3); Intermittent Explosive Disorder-IR (n = 7); Somatoform Disorder (n = 2); Eating Disorder (n = 1). Lifetime Axis I disorders were as follows: Any Mood Disorder (n = 12): major depression (n = 8), dythymia (n = 4), depressive disorder-NOS (n = 2); Any Anxiety Disorder (n = 4), i.e., phobic (n = 3), and non-phobic (n = 2) anxiety disorder; Alcohol Dependence (n = 6), Drug Dependence (n = 5); Intermittent Explosive Disorder-IR: (n = 7); Adjustment Disorder (n = 2); Eating Disorder (n = 2); Somatoform Disorder (n = 2). In addition to meeting criteria for Axis I and/or II disorders, most (79%) PD subjects reported: a) history of psychiatric treatment (64%) or, b) history of behavioral disturbance during which the subject, or others, thought they should have sought mental health services but did not (15%).

2.3. Assessment of aggression and impulsivity

Aggression measures included the Aggression score from the Life History of Aggression assessment (LHA; Coccaro et al., 1997) and the Aggression Factor score from the Buss–Durkee Hostility Inventory (BDHI; Buss and Durkee, 1957). LHA Aggression reflects a subject’s history of actual aggressive behavior whereas BDHI Aggression reflects a subject’s self-assessment of his or her tendency to be aggressive in given situations. Impulsivity measures included the Impulsiveness Scale from the Eysenck Personality Questionnaire II (EPQ-II; Eysenck and Eysenck, 1977) and the Barratt Impulsiveness Scale-Version 11 (BIS-11; Patton et al., 1995). These measures reflect a subject’s self-assessment of how impulsive he or she is. History of suicidal behavior was assessed during the diagnostic work-up as previously described (Bunce et al., 2005). Other assessments used in this study include the EPQ-I scales Neuroticism, Psychoticism, and Extraversion (Eysenck and Eysenck, 1975) and the remaining two scales from the EPQ-II (Venturesomeness and Empathy) as control dimensions of personality. Global function of subjects was assessed by the Global Assessment of Function scale (GAF, APA, 1994).

2.4. General preparation for study

No subject was taking any medical or psychotropic agent for at least four weeks at time of study and only six (16%) of the thirty-eight subjects (all PD subjects) had any lifetime exposure to psychotropic agents. Of the latter group, three PD subjects had lifetime exposure to three agents [antidepressant/benzodiazepine/stimulant (n = 1), antidepressant/benzodiazepine/sedative-hypnotic (n = 1), antidepressant/benzodiazepine/antipsychotic (n = 1)]; one had lifetime exposure to two agents (antidepressant/benzodiazepine); and two had lifetime exposure to only one agent [benzodiazepine (n = 1), sedative-hypnotic (n = 1)]. Subjects were instructed to follow a low monoamine diet for at least three (3) days prior to study (this was done to control for dietary contributions of tryptophan and tyrosine to CSF monoamine acid levels which were also available on these subjects). Subjects were also informed that initial and follow-up urine toxicology would be performed randomly just prior to study; illicit drug use was not detected in any subject reported herein. Females were all studied within the first ten days of the follicular phase of the menstrual cycle.

2.5. Lumbar puncture

Subjects reported to the Clinical Procedures Lab (CPL) at approximately 8:00 pm the evening before the lumbar puncture procedure. At approximately 11 pm subjects had a snack and were placed at rest in a supine position in a hospital bed in the CPL. Lumbar punctures were performed by a research neurologist in the morning hours after no less than 8 h of fasting and bedrest, under sterile technique, with the subject in the lateral decubitus position. A total of 20 cc of CSF was drawn in six aliquots. A single one cc aliquot representing the 19th cc of lumbar spinal fluid was used in this study for the assay of CSF Glutamate. All CSF samples were placed in polypropylene tubes and were frozen immediately at −70 °C until assay by high performance liquid chromatography (HPLC).

2.6. Assay of CSF glutamate

Prior to assay, samples were subjected to precolumn derivatization with o-phthaldialdehyde and β-mercaptoethanol. A portion of the mixture was then injected onto a Phenomenex Gemini 5 μm C18 150 mm × 4.6 mm column (maintained at 40 °C) using a Shimadzu system, consisting of a Model SIL-10ADVP autoinjector and Model RF-10AXL fluorescence detector (excitation and emission wavelengths set to 340 and 460 nm respectively). The gradient consisted of 40 mM sodium phosphate pH 7.8 (A) and a MeOH–acetonitrile–water mixture (B), progressing from 100% A to 100% B, with a flow rate of 1.0 ml/min. Glutamate concentrations were estimated from peak areas using Shimadzu CLASS-VP (Shimadzu Scientific Instruments, Columbia, MD) software. The lower limit of detection for this assay was .5 ng/ml; intra-/inter-assay coefficient of variation was <8.5%. Details on the assays of the other CSF variables reported in this paper (i.e., 5-Hydroxyindolacetic Acid; Homovanillic Acid; Vasopressin; Oxytocin; Neuropeptide Y; Substance P) can be found in the publications in which they were first reported (Coccaro et al., 1998, 2012a, 2012b; Coccaro and Lee, 2010; Lee et al., 2009).

2.7. Statistical analysis and data reduction

Comparisons of between group variables were performed by t-test, with correction for unequal variances where necessary, analysis of covariance (ANCOVA), and by X2 tests. Correlational analyses included Pearson correlation, partial correlation, and multiple regression. A two-tailed alpha value of .05 was used to denote statistical significance for all analyses. Basal lumbar CSF Glutamate levels were normally distributed. Finally, rather than use each of the aggression and impulsivity variables separately, composite variables for “aggression”, “impulsivity”, and “impulsive aggression” were created in a data-reduction step, as described previously (Coccaro et al., 2010). Composite variables were created because inter-correlations among the behavioral variables within domains were substantial (LHA Aggression with BDHI Aggression: r = .50; EPQ-II Impulsivity with BIS Impulsivity: r = .79; 25% shared variance for the aggression variables and 61% for the impulsivity variables). Composite variables were constructed by taking the average of each subject’s z-scores for the primary behavioral measures. The correlation between the composite variables for “aggression” and “impulsivity” was very large (r = .89; 79% of shared variance) indicating that this “impulsivity-aggression” variable more fully reflected the larger construct of impulsive aggression than either component alone.

3. Results

Demographic characteristics for the HV and PD subjects are displayed in Table 1. These groups did not differ in the distribution of gender or race or in age, height, weight, or in Hollinghead’s socioeconomic score. However, multiple regression of raw CSF Glutamate levels, as dependent variable, and age, gender, race, SES, height and weight as independent variables, revealed a nearly significant relationship between raw CSF Glutamate levels and these demographic/physical variables (R = .56, R2 = .18, F[6,31] = 2.33, p = .057). Accordingly, all analyses employed the residual value for the CSF Glutamate level after the removal of the variance associated with these variables. As expected, HV and PD subjects differed significantly in Aggression (BDHI), Impulsivity (BIS-11), and in GAF scores.

Table 1.

Sample characteristics.

Healthy volunteer (N = 10) Personality disorder (N = 28) P value
Demographic variables
Age (years) 32.0 ± 8.0 33.1 ± 7.5 .705a
Gender (M/F) 9/1 19/9 .236b
Race (white/non-white) 4/6 20/8 .127b
SES class score 33.4 ± 11.9 28.3 ± 10.6 .214a
Physical variables
Height (cm) 176.8 ± 9.0 173.4 ± 9.6 .337a
Weight (kg) 78.2 ± 9.6 73.1 ± 15.7 .351a
Behavioral variables
LHA aggression 4.9 ± 4.8 7.9 ± 5.4 .129a
BDHI aggression 13.7 ± 8.8 21.9 ± 10.6 .036a
EPQ-2 impulsivity 5.5 ± 4.4 7.4 ± 5.5 .347a
BIS-11 impulsivity 56.3 ± 6.3 87.1 ± 14.3 .037a
GAF score 83.4 ± 4.2 58.6 ± 10.3 .001a
Glutamate variables
Raw CSF level (ng/ml) 142.0 ± 50.5 116.1 ± 48.9 .162a
Adjusted CSF levelc .20 ± .68 −.07 ± .99 .427a
Open in a new tab
a

Significance level after t-test.

b

Significance level after Fischer Exact Test.

c

Adjusted for demographic and physical variables, see text.

3.1. CSF glutamate concentration and aggression and impulsivity

While PD and HV subjects did not differ in raw or adjusted CSF glutamate level (Table 1), adjusted CSF Glutamate levels correlated with composite aggression (r = .46, n = 38, p = .004), composite impulsivity (r = .32, n = 34, p = .062), and composite impulsive aggression (r = .38, n = 34, p = .026) in all subjects; Fig. 1. Positive correlations were seen with both PD and HV groups examined separately. In the PD group these correlations were: (a) composite aggression: r = .53, n = 28, p = .004; (b) composite impulsivity: r = .43, n = 25, p = .033; (c) composite impulsive aggression: r = .50, n = 25, p = .012. Correlations were similar in magnitude in the HV group but not statistically significant due to small sample size: (a) composite aggression: r = .58, n = 10, p = .082; (b) composite impulsivity: r = .43, n = 9, p = .249; (c) composite impulsive aggression: r = .57, n = 9, p = .109. Group differences in the magnitude of these correlations were not significant.

Fig. 1.

Fig. 1

Open in a new tab

(a) Correlation between residual CSF glutamate level (ng/ml) and composite aggression score in all subjects (Open Circles: Healthy Control Subjects; Closed Circles: Personality Disorder Subjects). (b) Correlation between residual CSF glutamate level (ng/ml) and composite impulsivity score in all subjects (Open Circles: Healthy Control Subjects; Closed Circles: Personality Disorder Subjects). (c) Correlation between residual CSF glutamate level (ng/ml) and composite impulsive Aggression score in all subjects (Open Circles: Healthy Control Subjects; Closed Circles: Personality Disorder Subjects).

3.2. CSF glutamate, IED-IR, and history of suicide attempt in the PD group

Among the PD group, seven subjects had a current diagnosis of IED-IR (IED+) and six had a life history of a suicide attempt (SA+). Adjusted CSF Glutamate levels were non-significantly higher as a function of IED (IED+: .25 ± .98 vs. IED−: −.20 ± .98, t26 = 1.20, p = .24) and non-significantly higher as a function of life history of suicide attempt (SA+: .25 ± .97 vs. SA−: −.16 ± 1.00, t26 = .89, p = .383). While both sets of differences represented “medium-sized” effects (d = .52 for IED; d = .41 for SA), the number of IED+, or SA+, subjects was too small for these differences to reach an alpha level of .05.

3.3. CSF glutamate and other Axis I and II diagnostic variables in the PD group

After correcting for multiple comparisons, no difference was observed in adjusted CSF Glutamate concentration as a function of current history of mood disorder or anxiety disorder; life history of mood, anxiety, alcohol or substance use dependent disorders; or as a function of Axis II Personality Disorder Cluster.

3.4. CSF glutamate and non-aggressive behavioral variables

Multiple regression analysis of the control personality variables (EPQ-I Neuroticism, Psychoticism, Extraversion, and EPQ-II Venturesomeness and Empathy) showed no significant relationship to adjusted CSF Glutamate levels in all subjects (F[5,30] = .87, p = .510) or in PD subjects examined alone (F[5,20] = .89, p = .504).

3.5. Relationship between CSF glutamate and other CSF variables previously associated with aggression in human studies

With the exception of CSF Substance P levels, adjusted CSF Glutamate levels correlated minimally (r ≤ .14, r2 ≤ .020, p > .40) with the CSF levels of other neurotransmitters associated with human aggression (5-Hydroxyindoleacetic Acid: r = .14, p = .41; Vasopressin: r = .11, p = .55; Homovanillic Acid: r = .09, p = .59; Neuropeptide Y: r = .09, p = .58; Oxytocin: r = .07, p = .69; Coccaro et al., 1998, 2011, 2012a, 2012b; Coccaro and Lee, 2010; Lee et al., 2009). In comparison, the correlation between adjusted CSF Glutamate and CSF Substance P levels was larger in magnitude, though not statistically significant (r = .27, r2 = .073, p = .13). Partial correlation, controlling for CSF Substance P levels, did not change the findings for all subjects (composite aggression: r = .44, p = .008; composite impulsivity: r = .36, p = .041; composite impulsive aggression: r = .41, p = .020) or for PD subjects examined alone (composite aggression: r = .51, p = .010; composite impulsivity: r = .50, p = .016; composite impulsive aggression: r = .55, p = .006).

4. Discussion

This is the first study to investigate the relationship between central nervous system glutamate levels and aggression and/or impulsivity in human subjects. We found statistically significant direct correlations between CSF Glutamate levels and composite measures of aggression, impulsivity, and impulsive aggression in all subjects and in PD subjects. While this sample is modest in size, these results are consistent with the pre-clinical literature, and provide initial support for a glutaminergic hypothesis of aggression in human subjects. This hypothesis posits that as glutaminergic transmission increases so does the risk of aggressive behavior in human subjects.

These findings were not due to any confounding demographic (e.g., age, gender, race, socioeconomic), or physical (e.g., height, weight), factors. These findings, also, cannot be accounted for by the presence of any Axis I disorder or to other dimensions of personality apart from impulsivity and aggression. As such, these data support the idea of restructuring PD diagnoses into relevant behavioral dimensions, at least with regard to impulsive aggression. Finally, these findings are unlikely to be accounted for by relationships between CSF Glutamate and CSF levels of other neurotransmitters that correlate with aggression in human subjects (Coccaro et al., 1998, 2012a, 2012b; Coccaro and Lee, 2010; Lee et al., 2009). Only minimally sized, non-significant, correlations were seen between CSF Glutamate levels and CSF levels reflecting central activity of serotonin, dopamine, vasopressin, oxytocin, or neuropeptide y. While the relationship between CSF Glutamate and CSF Substance P levels was of greater magnitude, statistical control of CSF Substance P levels did not change the statistical significance of the relationships between CSF Glutamate and Composite Aggression, Composite Impulsivity, or Composite Impulsive Aggression in all subjects, or in PD subjects alone.

No studies of CSF Glutamate levels and aggression in human subjects have been reported to date. However, CSF and Brain Glutamate levels have been investigated in subjects with other neuropsychiatric disorders such as schizophrenia, mood disorder, obsessive compulsive disorder, and pathological gambling. In schizophrenia, Kim et al. (1980) reported a reduction of CSF Glutamate in patients with schizophrenia. While this finding has not been consistently replicated, with several reports of no alteration in CSF Glutamate (Do et al., 1995; Gattaz et al., 1985; Perry, 1982; Tsai et al., 1998), at least one study reported an inverse correlation between CSF Glutamate and positive symptoms of schizophrenia, including hallucinatory behavior (Faustman et al., 1999). A more recent MR spectroscopy study, however, reported higher levels of glutamate in the associative-striatum of subjects with prodromal symptoms of schizophrenia and patients with first-episode psychosis (de la Fuente-Sandoval et al., 2011). Studies in treatment refractory bipolar patients (Frye et al., 2007a, 2007b) have reported low CSF Glutamate levels. This is in contrast to MR spectroscopy studies demonstrating elevated levels of glutamate, and related species, in the frontal lobe and basal ganglia (Castillo et al., 2000), dorsal lateral prefrontal cortex (Michael et al., 2003), left cingulate (Dager et al., 2004), and anterior cingulate (Frye et al., 2007a, 2007b) of bipolar patients. Similar MR spectroscopy studies in unipolar depressed patients have reported lower levels of glutamate and related species compared with controls (Auer et al., 2000; Mirza et al., 2004; Pfleiderer et al., 2003; Rosenberg et al., 2005). Recently, two studies report elevated CSF Glutamate levels in subjects with pathological gambling (Nordin et al., 2007) and in subjects with obsessive-compulsive disorder (Chakrabarty et al., 2005).

Consistent with our findings, elevated levels of aggression in subjects with pathological gambling (Afifi et al., 2010), and elevated levels of impulsivity in subjects with obsessive-compulsive disorder (Boisseau et al., 2012), have also been reported. Finally, while there have been no studies of CSF Glutamate levels in addictive disorders, a strong body of preclinical evidence suggests an important role for glutamate transmission and glutamate receptors in drug reward, reinforcement, and relapse (Reissner and Kalivas, 2010). Central glutaminergic neurons are found in behaviorally relevant cortico-limbic areas of the brain, with expression in the particularly relevant “attack” area of the hypothalamus (Hrabovszky et al., 2005). The capacity of CSF Glutamate to reflect central neuronal processes, such as activity in cortical and subcortical structures, is limited in the same way that most CSF biomarkers are limited. However, because levels of glutamate in the CSF are less than 20% of glutamate levels in the peripheral circulation (Abbott et al., 2010), and because the blood–brain-barrier actively moves glutamate from the interstitial fluid of the brain to the peripheral circulation (Helms et al., 2012), CSF Glutamate levels likely reflect brain and central nervous system glutamate without much influence from levels in the periphery.

These data are consistent with the preclinical literature in suggesting a facilitating role for glutamate in aggressive behavior (Comai et al., 2012). A synthesis of this work posits that glutamate can stimulate NMDA, AMPA, and metabotropic receptors in the amygdala, medial hypothalamus, and periaqueductal gray to enhance defensive-rage behavior. Stimulation of each of these structures by glutamate has been shown to produce this behavior (Bandler, 1984; Lu et al., 1992; Schubert et al., 1996). There is also evidence that stimulation by glutamate of the amygdala or the medial hypothalamus triggers defensive rage from the periaqueductal gray (Shaikh et al., 1994). Finally, genetic knockout of selected glutaminergic receptors (Duncan et al., 2004; Vekovischeva et al., 2004, 2007), or administration of agents that dampen glutaminergic activity suppresses aggressive behavior in lower mammals (Navarro et al., 2006, 2008).

The strengths of this study include a well characterized sample, multiple validated measures of aggression and impulsivity, and a standardized approach to drug-free status, subject activity, and dietary intake to minimize the effect of extraneous factors on CSF concentrations of the various neurotransmitters assessed in this study. Limitations include the fact that this is a cross-sectional study and no causal conclusions can be made from associative and correlational analyses. This is especially important because only a small percentage of measured glutamate serves as a neurotransmitter and causal conclusions may only be supported by experimental studies in human subjects. Accordingly, experimental study in which agents that reduce glutaminergic activity are tested for anti-aggressive properties in human subjects either in laboratory studies of human aggression, and/or in clinical trials, will be necessary to demonstrate a direct role of central glutamate in human aggression. Second, it is possible that these findings will be smaller, or non-existent, in a larger sample. Third, ascertainment of subjects may limit the generalizability of these findings in that these involved subjects who volunteered for a research study, rather than for clinical treatment. However, nearly eighty percent of the personality disordered subjects reported a past history of psychiatric treatment, or of having episodes of behavioral disturbance for which they, or others, thought they should have sought mental health services but did not. If so, most of the personality disordered subjects in this report may be similar to subjects who would have been recruited from a clinical setting.

In summary, we report a relationship between CSF Glutamate concentration and aggression and impulsivity in human subjects, particularly in personality disordered subjects. This relationship was not accounted for by any of the other factors studied such as psychiatric disorder or general personality factors other than aggression and/or impulsivity. These data are in line with the hypothesized central role glutamate plays in regulation of impulsive aggression. Given that disorders of impulsive aggression, display a 2–3% one year prevalence rate in the U.S. (Kessler et al., 2006), and that currently available psychotropic treatments bring less than 50% of those treated into remission (Coccaro et al., 2009), additional strategies for the examination and intervention of impulsive aggression in human subjects are needed.

Acknowledgments

Role of funding source

This work was supported in part by grants from the National Institute of Mental Health: RO-1 MH46848 and RO-1 MH80109 (Dr. Coccaro) and R01 DA09397 (Dr. Vezina).

The authors thank Nancy Bubula and Margaret Wieczorek for their technical assistance in this work.

Footnotes

Conflict of interest statement

Dr. Coccaro reports being on the Scientific Advisory Board of Azevan Pharmaceuticals, Inc. and being a current recipient of grants from the NIMH. Dr. Lee reports being a past recipient of a research grant from Azevan Pharmaceuticals, Inc. Dr. Vezina reports no conflicts of interest regarding this work.

References

  1. Abbott NJ, Patabendige AA, Dolman DE, Yusof SR, Begley DJ. Structure and function of the blood-brain barrier. Neurobiology of Disease. 2010;37:13–25. doi: 10.1016/j.nbd.2009.07.030. [DOI] [PubMed] [Google Scholar]
  2. Adamczyk A, Mejias R, Takamiya K, Yocum J, Krasnova IN, Calderon J, et al. GluA3-deficiency in mice is associated with increased social and aggressive behavior and elevated dopamine in striatum. Behavioral Brain Research. 2012;229:265–72. doi: 10.1016/j.bbr.2012.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Afifi TO, Brownridge DA, MacMillan H, Sareen J. The relationship of gambling to intimate partner violence and child maltreatment in a nationally representative sample. Journal of Psychiatric Research. 2010;44:331–7. doi: 10.1016/j.jpsychires.2009.07.010. [DOI] [PubMed] [Google Scholar]
  4. Ago Y, Araki R, Yano K, Kawasaki T, Chaki S, Nakazato A, et al. The selective metabotropic glutamate 2/3 receptor agonist MGS0028 reverses isolation rearing-induced abnormal behaviors in mice. Journal of Pharmacological Science. 2012;118:295–8. doi: 10.1254/jphs.11200sc. [DOI] [PubMed] [Google Scholar]
  5. American Psychiatric Association. Diagnostic and statistical manual. 4. Washington, D.C: American Psychiatric Press; 1994. (DSM-IV) [Google Scholar]
  6. Auer DP, Putz B, Kraft E, Lipinski B, Schill J, Holsboer F. Reduced glutamate in the anterior cingulate cortex in depression: an in vivo proton magnetic resonance spectroscopy study. Biological Psychiatry. 2000;47:305–13. doi: 10.1016/s0006-3223(99)00159-6. [DOI] [PubMed] [Google Scholar]
  7. Bandler R. Identification and midbrain periaqueductal grey neurones mediating aggressive and defensive behavior by intracerebral microinjections of excitatory amino acids. In: Bandler R, editor. Modulation of sensorimotor activity during alterations in behavior states. New York: Alan R. Liss, Inc; 1984. pp. 369–91. [Google Scholar]
  8. Beart PM, Nicolopoulos LS, West DC, Headley PM. An excitatory amino acid projection from ventromedial hypothalamus to periaqueductal gray in the rat: autoradiographic and electrophysiology evidence. Neuroscience Letters. 1998;85:205–11. doi: 10.1016/0304-3940(88)90352-7. [DOI] [PubMed] [Google Scholar]
  9. Beart PM, Summers RJ, Stephenson JA, Cook CJ, Christie MJ. Exciatory amino acid projections to the periaqueductal gray in the rat: a retrograde transport study utilizing d[3H]aspartate and [3H]GABA. Neuroscience. 1990;34:163–76. doi: 10.1016/0306-4522(90)90310-z. [DOI] [PubMed] [Google Scholar]
  10. Beitz AJ. Possible origin of glutamatergic projections to the midbrain periaqueductal gray and deep layer of the superior colliculus of the rat. Brain Research Bulletin. 1989;23:25–35. doi: 10.1016/0361-9230(89)90159-7. [DOI] [PubMed] [Google Scholar]
  11. Boisseau CL, Thompson-Brenner H, Caldwell-Harris C, Pratt E, Farchione T, Harrison Barlow D. Behavioral and cognitive impulsivity in obsessive-compulsive disorder and eating disorders. Psychiatry Research. 2012;200:1060–6. doi: 10.1016/j.psychres.2012.06.010. [DOI] [PubMed] [Google Scholar]
  12. Brodkin ES, Goforth SA, Keene AH, Fossella JA, Silver LM. Identification of quantitative trait loci that affect aggressive behavior in mice. Journal of Neuroscience. 2002;22:1165–70. doi: 10.1523/JNEUROSCI.22-03-01165.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bunce SC, Noblett KL, McCloskey MS, Coccaro EF. High prevalence of personality disorders among healthy volunteers for research: implications for control group bias. Journal of Psychiatric Research. 2005;39:421–30. doi: 10.1016/j.jpsychires.2004.09.005. [DOI] [PubMed] [Google Scholar]
  14. Buss AH, Durkee A. An inventory for assessing different kinds of hostility. Journal of Consulting Psychology. 1957;21:343–8. doi: 10.1037/h0046900. [DOI] [PubMed] [Google Scholar]
  15. Carlson GC. Glutamate receptor dysfunction and drug targets across models of autism spectrum disorders. Pharmacology, Biochemistry, and Behavior. 2012;100:850–4. doi: 10.1016/j.pbb.2011.02.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Castillo M, Kwock L, Courvoisie H, Hooper SR. Proton MR spectroscopy in children with bipolar affective disorder: preliminary observations. American Journal of Neuroradiology. 2000;21:832–8. [PMC free article] [PubMed] [Google Scholar]
  17. Chakrabarty K, Bhattacharyya S, Christopher R, Khanna S. Glutamatergic dysfunction in OCD. Neuropsychopharmacology. 2005;30:1735–40. doi: 10.1038/sj.npp.1300733. [DOI] [PubMed] [Google Scholar]
  18. Coccaro EF. Intermittent Explosive Disorder: Development of integrated research criteria for diagnostic and statistical manual-fifth edition. Comprehensive Psychiatry. 2011;52:119–25. doi: 10.1016/j.comppsych.2010.05.006. [DOI] [PubMed] [Google Scholar]
  19. Coccaro EF. Intermittent explosive disorder as a disorder of impulsive aggression for DSM-5. American Journal of Psychiatry. 2012;169:577–88. doi: 10.1176/appi.ajp.2012.11081259. [DOI] [PubMed] [Google Scholar]
  20. Coccaro EF, Berman ME, Kavoussi RJ. Assessment of life history of aggression: development and psychometric characteristics. Psychiatry Research. 1997;73:147–57. doi: 10.1016/s0165-1781(97)00119-4. [DOI] [PubMed] [Google Scholar]
  21. Coccaro EF, Kavoussi RK, Hauger RL, Cooper TB, Ferris CF. Cerebrospinal fluid vasopressin: Correlates with aggression and serotonin function in personality disordered subjects. Archives of General Psychiatry. 1998;55:708–14. doi: 10.1001/archpsyc.55.8.708. [DOI] [PubMed] [Google Scholar]
  22. Coccaro EF, Lee R. Cerebrospinal fluid 5-hydroxyindolacetic acid and homovanillic acid: Reciprocal relationships with impulsive aggression in human subjects. Journal of Neural Transmission. 2010;117:241–8. doi: 10.1007/s00702-009-0359-x. [DOI] [PubMed] [Google Scholar]
  23. Coccaro EF, Lee R, Kavoussi RJ. A double-blind, randomized, placebo-controlled trial of fluoxetine in patients with intermittent explosive disorder. Journal of Clinical Psychiatry. 2009;70:653–62. doi: 10.4088/JCP.08m04150. [DOI] [PubMed] [Google Scholar]
  24. Coccaro EF, Lee R, Kavoussi RJ. Aggression, suicidality, and intermittent explosive disorder: serotonergic correlates in personality disorder and healthy control subjects. Neuropsychopharmacology. 2010;35:435–44. doi: 10.1038/npp.2009.148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Coccaro EF, Lee R, Owens MJ, Kinkead B, Nemeroff CB. Cerebrospinal fluid substance p-like immunoreactivity correlates with aggression in personality disordered subjects. Biological Psychiatry. 2012a;72:238–43. doi: 10.1016/j.biopsych.2012.02.023. [DOI] [PubMed] [Google Scholar]
  26. Coccaro EF, Lee R, Liu T, Mathé AA. Cerebrospinal fluid neuropeptide Y-like immunoreactivity correlates with impulsive aggression in human subjects. Biological Psychiatry. 2012b;72:997–1003. doi: 10.1016/j.biopsych.2012.07.029. [DOI] [PubMed] [Google Scholar]
  27. Comai S, Tau M, Gobbi G. The psychopharmacology of aggressive behavior: a translational approach part 1: neurobiology. Journal of Clinical Psychopharmacology. 2012;32:83–94. doi: 10.1097/JCP.0b013e31823f8770. [DOI] [PubMed] [Google Scholar]
  28. Dager SR, Friedman SD, Parow A, Demopulos C, Stoll AL, Lyoo IK, et al. Brain metabolic alterations in medication-free patients with bipolar disorder. Archives of General Psychiatry. 2004;61:450–8. doi: 10.1001/archpsyc.61.5.450. [DOI] [PubMed] [Google Scholar]
  29. de la Fuente-Sandoval C, León-Ortiz P, Favila R, Stephano S, Mamo D, Ramírez-Bermúdez J, et al. Higher levels of glutamate in the associative-striatum of subjects with prodromal symptoms of schizophrenia and patients with first-episode psychosis. Neuropsychopharmacology. 2011;36:1781–91. doi: 10.1038/npp.2011.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Do KQ, Lauer CJ, Schreiber W, Zollinger M, Gutteck-Amsler U, Cuenod M, et al. γ-Glutamylglutamine and taurine concentrations are decreased in the cerebro-spinal fluid of drug-naive patients with schizophrenic disorders. Journal of Neurochemistry. 1995;65:2652–62. doi: 10.1046/j.1471-4159.1995.65062652.x. [DOI] [PubMed] [Google Scholar]
  31. Duncan GE, Moy SS, Perez A. Deficits in sensorimotor gating and tests of social behavior in a genetic model of reduced NMDA receptor function. Behaviorial Brain Research. 2004;153:507–19. doi: 10.1016/j.bbr.2004.01.008. [DOI] [PubMed] [Google Scholar]
  32. Eysenck HJ, Eysenck SBG. Manual of the Eysenck Personality Questionnaire. London: Hodder and Stoughton; 1975. [Google Scholar]
  33. Eysenck SBG, Eysenck HJ. The place of impulsiveness in a dimensional system of personality description. Br J Soc Clin Psychology. 1977;16:57–68. doi: 10.1111/j.2044-8260.1977.tb01003.x. [DOI] [PubMed] [Google Scholar]
  34. Faustman WO, Bardgett M, Faull KF, Pfefferbaum A, Csernansky JG. Cerebrospinal fluid glutamate inversely correlates with positive symptom severity in unmedicated male schizophrenic/schizoaffective patients. Biological Psychiatry. 1999;45:68–75. doi: 10.1016/s0006-3223(98)00207-8. [DOI] [PubMed] [Google Scholar]
  35. Frye MA, Tsai GE, Huggins T, Coyle JT, Post RM. Low cerebrospinal fluid glutamate and glycine in refractory affective disorder. Biological Psychiatry. 2007;61:162–6. doi: 10.1016/j.biopsych.2006.01.024. [DOI] [PubMed] [Google Scholar]
  36. Frye MA, Watzl J, Banakar S, O’Neill J, Mintz J, Davanzo P, et al. Increased anterior cingulate/medial prefrontal cortical glutamate and creatine in bipolar depression. Neuropsychopharmacology. 2007;32:2490–9. doi: 10.1038/sj.npp.1301387. [DOI] [PubMed] [Google Scholar]
  37. Gattaz WF, Gasser T, Beckmann H. Multidimensional analysis of the concentrations of 17 substances in the CSF of schizophrenics and controls. Biological Psychiatry. 1985;20:360–6. doi: 10.1016/0006-3223(85)90038-1. [DOI] [PubMed] [Google Scholar]
  38. Helms HC, Madelung R, Waagepetersen HS, Nielsen CU, Brodin B. In vitro evidence for the brain glutamate efflux hypothesis: brain endothelial cells cocultured with astrocytes display a polarized brain-to-blood transport of glutamate. Glia. 2012;60:882–93. doi: 10.1002/glia.22321. [DOI] [PubMed] [Google Scholar]
  39. Hrabovszky E, Halász J, Meelis W, Kruk MR, Liposits Z, Haller J. Neurochemical characterization of hypothalamic neurons involved in attack behavior: glutamatergic dominance and co-expression of thyrotropin-releasing hormone in a subset of glutamatergic neurons. Neuroscience. 2005;133:657–66. doi: 10.1016/j.neuroscience.2005.03.042. [DOI] [PubMed] [Google Scholar]
  40. Hu W, Ondrejcak T, Rowan MJ. Glutamate receptors in preclinical research on Alzheimer’s disease: Update on recent advances. Pharmacology, Biochemistry, and Behavior. 2012;100:855–62. doi: 10.1016/j.pbb.2011.04.013. [DOI] [PubMed] [Google Scholar]
  41. Kessler RC, Coccaro EF, Fava M, Jaeger S, Jin R, Walters E. The prevalence and correlates of DSM-IV intermittent explosive disorder in the National Comorbidity Survey Replication. Archives of General Psychiatry. 2006;63:669–78. doi: 10.1001/archpsyc.63.6.669. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Kim JS, Kornhuber HH, Schmid-Burgk W, Holzmuller B. Low cerebrospinal fluid glutamate in schizophrenic patients and a new hypothesis on schizophrenia. Neuroscience Letters. 1980;20:379–82. doi: 10.1016/0304-3940(80)90178-0. [DOI] [PubMed] [Google Scholar]
  43. Lee R, Ferris C, Van de Kar LD, Coccaro EF. Cerebrospinal fluid oxytocin, life history of aggression, and personality disorder. Psychoneuroendocrinology. 2009;34:1567–73. doi: 10.1016/j.psyneuen.2009.06.002. [DOI] [PubMed] [Google Scholar]
  44. Lin CH, Lane HY, Tsai GE. Glutamate signaling in the pathophysiology and therapy of schizophrenia. Pharmacology, Biochemistry, and Behavior. 2012;100:665–77. doi: 10.1016/j.pbb.2011.03.023. [DOI] [PubMed] [Google Scholar]
  45. Lu CL, Shaikh MB, Siegel A. Role of NMDA receptors in hypothalamic facilitation of feline defensive rage elicited from the midbrain periaqueductal gray. Brain Research. 1992;581:123–32. doi: 10.1016/0006-8993(92)90351-9. [DOI] [PubMed] [Google Scholar]
  46. Machado-Vieira R, Ibrahim L, Henter ID, Zarate CA., Jr Novel glutamatergic agents for major depressive disorder and bipolar disorder. Pharmacology, Biochemistry, and Behavior. 2012;100:678–87. doi: 10.1016/j.pbb.2011.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Michael N, Erfurth A, Ohrmann P, Goessling M, Arolt V, Heindel W, et al. Acute mania is accompanied by elevated glutamate/glutamine levels within the left dorsolateral prefrontal cortex. Psychopharmacology. 2003;168:344–6. doi: 10.1007/s00213-003-1440-z. [DOI] [PubMed] [Google Scholar]
  48. Mirza Y, Tang J, Russell A, Banerjee S, Bhandari R, Ivey J. Reduced anterior cingulate cortex glutamatergic concentrations in childhood major depression. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43:341–8. doi: 10.1097/00004583-200403000-00017. [DOI] [PubMed] [Google Scholar]
  49. Navarro JF, Postigo D, Martín M, Burón E. Antiaggressive effects of MPEP, a selective antagonist of mGlu5 receptors, in agonistic interactions between male mice. European Journal of Pharmacology. 2006;551:67–70. doi: 10.1016/j.ejphar.2006.08.055. [DOI] [PubMed] [Google Scholar]
  50. Navarro JF, de Castro V, Martin-Lopez M. JNJ16259685, a selective mGlu(1) antagonist, suppresses isolation-induced aggression in male mice. European Journal of Pharmacology. 2008;586:217–20. doi: 10.1016/j.ejphar.2008.02.070. [DOI] [PubMed] [Google Scholar]
  51. Navarro JF, de Castro V, Martín-López M. Behavioural profile of selective ligands for mGlu7 and mGlu8 glutamate receptors in agonistic encounters between mice. Psicothema. 2009;21:475–9. [PubMed] [Google Scholar]
  52. Niciu MJ, Kelmendi B, Sanacora G. Overview of glutamatergic neurotransmission in the nervous system. Pharmacology, Biochemistry, and Behavior. 2012;100:656–64. doi: 10.1016/j.pbb.2011.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Nordin C, Gupta RC, Sjödin I. Cerebrospinal fluid amino acids in pathological gamblers and healthy controls. Neuropsychobiology. 2007;56:152–8. doi: 10.1159/000115782. [DOI] [PubMed] [Google Scholar]
  54. Olive MF, Cleva RM, Kalivas PW, Malcolm RJ. Glutamatergic medications for the treatment of drug and behavioral addictions. Pharmacology, Biochemistry, and Behavior. 2012;100:801–10. doi: 10.1016/j.pbb.2011.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Patton JH, Stanford MS, Barratt ES. Factor structure of the Barratt impulsiveness scale. Journal of Clinical Psychology. 1995;51:768–74. doi: 10.1002/1097-4679(199511)51:6<768::aid-jclp2270510607>3.0.co;2-1. [DOI] [PubMed] [Google Scholar]
  56. Perry TL. Normal cerebrospinal fluid and brain glutamate levels in schizophrenia do not support the hypothesis of glutamatergic neuronal dysfunction. Neuroscience Letters. 1982;28:81–5. doi: 10.1016/0304-3940(82)90212-9. [DOI] [PubMed] [Google Scholar]
  57. Pfleiderer B, Michael N, Erfurth A, Ohrmann P, Hohmann U, Wolgast M. Effective electroconvulsive therapy reverses glutamate/glutamine deficit in the left anterior cingulum of unipolar depressed patients. Psychiatry Research. 2003;3:185–92. doi: 10.1016/s0925-4927(03)00003-9. [DOI] [PubMed] [Google Scholar]
  58. Reissner KJ, Kalivas PW. Using glutamate homeostasis as a target for treating addictive disorders. Behavioral Pharmacology. 2010;21:514–22. doi: 10.1097/FBP.0b013e32833d41b2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Riaza Bermudo-Soriano C, Perez-Rodriguez MM, Vaquero-Lorenzo C, Baca-Garcia E. New perspectives in glutamate and anxiety. Pharmacology, Biochemistry, and Behavior. 2012;100:752–74. doi: 10.1016/j.pbb.2011.04.010. [DOI] [PubMed] [Google Scholar]
  60. Rosenberg DR, MacMaster FP, Mirza Y, Smith JM, Easter PC, Banerjee SP. Reduced anterior cingulate glutamate in pediatric major depression: a magnetic resonance spectroscopy study. Biological Psychiatry. 2005;58:700–4. doi: 10.1016/j.biopsych.2005.05.007. [DOI] [PubMed] [Google Scholar]
  61. Schubert K, Shaikh MB, Siegel A. NMDA receptors in the midbrain periaqueductal gray mediate hypothalamically evoked hissing behavior in the cat. Brain Research. 1996;726:80–90. [PubMed] [Google Scholar]
  62. Shaikh MB, Schubert K, Siegel A. Basal amygdaloid facilitation of midbrain periaqueductal gray elicited defensive rage behavior in the cat is mediated through NMDA receptors. Brain Research. 1994;635:187–95. doi: 10.1016/0006-8993(94)91438-9. [DOI] [PubMed] [Google Scholar]
  63. Tsai G, van Kammen DP, Chen S, Kelley ME, Grier A, Coyle JT. Glutamatergic neurotransmission involves structural and clinical deficits of schizophrenia. Biological Psychiatry. 1998;44:667–74. doi: 10.1016/s0006-3223(98)00151-6. [DOI] [PubMed] [Google Scholar]
  64. Vekovischeva OY, Aitta-aho T, Echenko O, Kankaanpää A, Seppälä T, Honkanen A, et al. Reduced aggression in AMPA-type glutamate receptor GluR-A subunit-deficient mice. Genes, Brain, and Behavior. 2004;3:253–65. doi: 10.1111/j.1601-1848.2004.00075.x. [DOI] [PubMed] [Google Scholar]
  65. Vekovischeva OY, Aitta-aho T, Verbitskaya E, Sandnabba K, Korpi ER. Acute effects of AMPA-type glutamate receptor antagonists on intermale social behavior in two mouse lines bidirectionally selected for offensive aggression. Pharmacology, Biochemistry, and Behavior. 2007;87:241–9. doi: 10.1016/j.pbb.2007.04.020. [DOI] [PubMed] [Google Scholar]

RESOURCES