Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Dec 15.
Published in final edited form as: Biol Psychiatry. 2007 Aug 13;62(12):1396–1404. doi: 10.1016/j.biopsych.2007.05.025

Brain metabolite abnormalities in the white matter of elderly schizophrenic subjects: implication for glial dysfunction

Linda Chang 1, Joseph Friedman 2, Thomas Ernst 1, Kai Zhong 3, Nicholas D Tsopelas 4, Kenneth Davis 2
PMCID: PMC2222890  NIHMSID: NIHMS35733  PMID: 17693392

Abstract

Background

Abnormalities in the white matter of the brain may occur in individuals with schizophrenia as well as with normal aging. Therefore, elderly schizophrenic patients may suffer further cognitive decline as they age. This study determined whether elderly schizophrenia participants, especially those with declined cognitive function (CDR>1), show white matter metabolite abnormalities on proton magnetic resonance spectroscopy (1H MRS), and whether there are group differences in age-dependent changes in these brain metabolites.

Method

23 elderly schizophrenic and 22 comparison participants fulfilling study criteria were enrolled. Localized, short echo-time 1H MRS at 4 Tesla was used to assess neurometabolite concentrations in several white matter regions.

Results

Compared to healthy subjects, schizophrenic participants had lower N-acetyl compounds (NA, −12.6%, p=0.0008), lower myoinositol (MI, −16.4%, p=0.026) and higher glutamate+glutamine (GLX, +28.7%, p=0.0016) concentrations across brain regions. Schizophrenic participants with CDR≥1 showed the lowest NA in the frontal and temporal regions compared to controls. Interactions between age and schizophrenia status on total creatine (CR) and choline-containing compounds (CHO) were observed; only schizophrenic participants showed age-related decreases of these two metabolites in the right frontal region.

Conclusion

Decreased NA in these white matter brain regions likely reflects reduced neuronal content associated with decreased synapses and neuronal cell volumes. The elevated GLX, if reflecting elevated glutamate, could result from excess neuronal glutamate release or glial dysfunction in glutamate re-uptake. The decreased MI in participants with schizophrenia suggests decreased glial content or dysfunctional glia, which might result from glutamate-mediated toxicity.

Keywords: Schizophrenia, glutamate, myoinositol, spectroscopy, white matter

Emerging evidence suggests that abnormalities in oligodendroglia or myelination can lead to white matter pathology that may contribute to the disease processes of schizophrenia (1). Brain tissues of schizophrenic patients demonstrated ultra-structural alterations of the myelin, including inclusions and loss of myelin compactness (2), as well as decreased number or death of oligodendroglia (3). In addition, DNA microarray analysis of post mortem tissue from the dorsolateral prefrontal cortex demonstrated that six genes whose expression is enriched in myelin-forming oligodendrocytes were down-regulated in the schizophrenic compared to control subjects (4). These six genes were implicated in the formation and maintenance of myelin sheaths, which are critical for efficient axonal signal propagation and provide extrinsic trophic signals that affect the development and long-term survival of axons (1).

Little is known regarding abnormalities in the myelin or white matter in schizophrenia, and whether such abnormalities affect cognitive function. The cognitive and functional deficits commonly seen in younger schizophrenic patients appear to worsen in some who are becoming elderly (>60 years of age). Despite the profound functional and cognitive decline in some of these elderly schizophrenic patients, neuropathological studies do not show the pathological hallmark of other dementia syndromes, such as Alzheimer’s disease, multi-infarcts, or Lewy body dementia. Since white matter and myelin changes also occur with normal aging (5, 6), degenerative processes during normal aging may exacerbate the pathophysiology of schizophrenia in elderly schizophrenic patients.

Furthermore, recent data suggest dysfunctional interactions between the glutamatergic and dopaminergic systems may ultimately lead to excess release of glutamate (7, 8), in particular in schizophrenic patients with cognitive impairment and negative symptoms (9). The important pathophysiologic role of glutamate is further bolstered by the discovery of many susceptibility genes for schizophrenia that act on glutamatergic transmission (10). Since astroglia is essential for glutamate homeostasis, dysfunctional uptake of glutamate by astroglia also might contribute to elevated extracellular glutamate and hence excitotoxicity to neurons and glia in schizophrenic subjects. Recent in vivo measurements documented elevated glutamine in the left anterior cingulate and thalamus of never-treated schizophrenic patients (11), as well as elevated glutamate in prefrontal and hippocampal regions in chronic schizophrenic patients who were treated with medications (12). However, little is known regarding changes in glutamate levels in the white matter and in elderly schizophrenic patients.

This study used proton magnetic resonance spectroscopy (1H MRS) at 4 Tesla to assess white matter abnormalities in the brains of elderly schizophrenic patients, including those with declined cognitive function (CDR ≥1). The study also aimed to determine whether elderly schizophrenic participants show different age-related changes of neurometabolites in the white matter compared to non-schizophrenic healthy comparison subjects. We hypothesized that elderly schizophrenic subjects, especially those with cognitive decline, would show decreased myoinositol (MI), a glial marker, due to possible glial dysfunction. Based on prior studies (12), we proposed that glutamate+glutamine (GLX) would be elevated due to a hyperglutamatergic state and dysfunctional glial reuptake of glutamate. The glial dysfunction also may contribute to neuronal dysfunction, and hence lead to decreased N-acetylaspartate, or N-acetyl compounds (NA, a neuronal marker). Although white matter diseases typically are associated with elevated choline compounds (CHO) and total creatine (CR), a meta-analysis of over 25 1H MRS studies of schizophrenic subjects showed no significant changes in either CR or CHO in the white matter (13); therefore, we do not expected significant changes in either one of these metabolites.

Methods

Participants

Twenty-three elderly participants with a DSM IV diagnosis of schizophrenia, including 3 with schizoaffective disorder based on our structured diagnostic interview (14), were enrolled. Fourteen of these participants had mild cognitive and functional decline (Clinical Dementia Rating (15) or CDR < 1) and 9 had moderate to severe decline (CDR ≥ 1). Participants with schizophrenia were initially screened and recruited from Pilgrim Psychiatric Center inpatient and outpatient facilities located in Brentwood, N.Y., and the Hudson Valley Veterans Affairs Medical Center inpatient psychiatric unit, Montrose, N.Y. An independent consent auditor assessed and ensured each participant’s capacity to provide informed consent. The subjects signed written consents approved at the institutions where they were recruited (Pilgram Hospital and Hudson Valley Veterans Affairs Medical Center) and from Brookhaven National Laboratory where the imaging studies were performed. Twenty-two healthy elderly comparison subjects from the local community also were recruited and enrolled in the study. Each participant underwent a neuropsychiatric evaluation, screening laboratory studies (including complete blood count, routine chemistry, liver enzymes, and thyroid function tests) and urine toxicology screen. Schizophrenic participants were included if they had: 1) DSM IV diagnosis of schizophrenia, with any subtype or schizoaffective disorder; 2) Clinical Dementia Rating score <2; 3) age ≥ 60 years; 4) male or female; 5) any ethnicity/race; 6) on medication therapy for schizophrenia; 7) willing and able to give informed consent (as assessed by a consent auditor independent from the study). Non-schizophrenic comparison participants were included only if they had: 1) no DSM IV Axis I diagnoses of any type; 2) Clinical Dementia Rating score < 1; 3) age ≥ 60 years; 4) male or female; 5) any ethnicity/race; and 6) willing and able to give informed consent. Exclusion criteria for all participants included: 1) other illness or factors present which may have caused psychiatric symptoms and/or cognitive impairments or decline; 2) poor medical health (e.g. unstable coronary artery disease, uncontrolled diabetes); 3) axis II developmental disorder or functional impairment that would prevent cognitive testing; 4) history of drug or alcohol dependence (except for nicotine) or positive urine drug screen on the day of the scan (marijuana, cocaine, PCP, benzodiazapines, opiates, barbiturates and amphetamines), except those due to medications; 5) presence of metal objects that are contraindicated for MR scanning; 6) documented history of violent acts. In addition, participants with schizophrenia were excluded if they had:1) other Axis 1 diagnosis in addition to schizophrenia or schizoaffective disorder; 2) current episode of major depression or mania (if schizoaffective disorder). Furthermore, non-schizophrenic participants were excluded if they had: 1) history of significant psychiatric illness requiring hospitalization; or 2) significant uncorrected visual or hearing deficits, or functional impairment which may interfere with cognitive testing and assessment of psychiatric symptoms.

MRI and MRS

MRI and localized 1H MRS were performed on a Varian 4 Tesla scanner. MRI started with the acquisition of a sagittal scout image (gradient echo, TE/TR=6/50 ms, one slice of 5 mm thickness, 24 cm FOV, flip angle 40°), followed by a 3-D modified driven equilibrium Fourier transformed (MDEFT) sequence (16) (TE/TR = 8/15 ms, mixing time 1200 ms, 24×18×14.4 cm FOV, resolution 256×192×48) to screen for structural abnormalities. Lastly, a coronal hyperecho sequence (17) (TE/TR=68/8000 ms, echo train length = 16, 256×256 matrix size, 28 coronal slices, 0.86 × 0.86 mm in-plane resolution, 5 mm thickness, 1 mm gap, 2 min scan time) yielded images for assessments of potential white matter abnormalities and the prescription of voxel locations.

Water-suppressed MRS data were acquired using an optimized double spin echo sequence (PRESS) (18), with TE=30 ms, TR=3000 ms, 64 averages, and 2 kHz bandwidth. Additionally, the unsuppressed water signal for each voxel was acquired at 7 different echo times (TE=30, 50, 80, 100, 200, 500, 1000ms; TR=15 seconds, 1 repetition), with other scanning parameters identical to those used for spectrum acquisition. In each participant, up to 5 MRS voxels were prescribed (Figure 1). In order to avoid the inclusion of gray matter, the voxels tended to be smaller in the schizophrenic subjects than the controls for all regions due to their smaller white matter volumes: [frontal white matter: right: 4.9±1.5 vs. 4.3±0.8 mm3; left: 4.5±1.5 vs. 4.3±0.9 mm), occipital white matter (right or left: 5.4±2.9 vs. 4.4±1.5 mm3), and temporal white matter (right: 5.1±1.7 vs. 3.8±1.4 mm3; left: 4.9±1.9 vs. 3.9±1.0 mm3). Two of these elderly participants were unable to remain motionless during the MRS acquisitions and data had to be discarded; the remainder 43 subjects had acceptable data from both right and left frontal regions, 35 had right temporal, 40 had left temporal and 30 had right or left occipital regions. The total scan time averaged 75 minutes (range 45–120 minutes).

Figure 1.

Figure 1

Open in a new tab

Left: Coronal images showing voxel locations in the left and right frontal white matter, left and right temporal white matter, and left or right occipital white matter. Right: Representative MR spectra, including the LC Model fittings from each metabolite, from the right frontal white matter regions of a 63-year-old patient with schizophrenia and CDR=1 (top) and a healthy 66-year-old comparison participant (bottom). Note relatively lower N-acetyl compounds (NA) and myo-inositol (MI) but elevated glutamate+glutamine (GLU+GLN) and elevated choline compounds (CHO) in the schizophrenic participant compared to the control participant. MM = macromolecules

The amplitudes of the unsuppressed water signals (7 TE values) were fitted to a double-exponential T2 decay model (19). The shorter of the two T2 components (T2 approximately 70 ms) represents water from brain parenchyma, whereas the long T2-component (T2 greater than 1000 ms) reflects water from CSF. The amplitude of the parenchymal water signal, corrected for T2, was used as an internal reference to calculate metabolite concentrations, which are insensitive to coil loading, to changes in the water T2, and to the percentage of CSF in each voxel (19, 20). The metabolite concentrations of NA, CR, CHO, MI, and GLX were determined in the LC Model Program (21), using a basis data set that included the spectra of alanine, aspartate, CHO, CR, GABA, glucose, glutamate, glutamine, glycine, lactate, MI, NAA, NAAG, taurine, and macromolecules. After fitting the separate contributions of glutamate and glutamine, GLX was defined (by LCModel) as the sum of glutamate plus glutamine. Spectra were included in the analysis only if the CR line-width and Cramer-Rao bounds, as determined by LC Model, were below 0.1 ppm and 20% for frontal and occipital white matter, and 0.13 ppm and 25% for temporal white matter.

Statistical analyses

Statistical analyses were performed in StatView (version 5.0.1, SAS Institute Inc., Cary, NC). Since there were no differences between hemisphere or brain regions from the analyses of covariance (ANCOVA) on the brain metabolites, in order to minimize multiple comparisons, metabolites in the different brain regions were averaged for each metabolite. Unpaired t-tests or the analysis of variance (ANOVA) were then used to compare differences between comparison and schizophrenic subjects for the average concentration of each metabolite; group differences for individual brain regions were also shown in Figures 2 & 3 with their respective p-values from unpaired t-tests. Correction for multiple comparisons was made using an improved Bonferroni procedure, which is a procedure that uses a rank-ordering of significance values (22). To assess possible interactions between aging and schizophrenia on cerebral metabolite levels, ANCOVA were performed, with disease status as a categorical variable and age as a covariate (Figure 4). A type I error probability ≤ 0.05 was used to determine statistical significance.

Figure 2.

Figure 2

Open in a new tab

Metabolite levels in all brain regions. Relative to comparison participants, those with schizophrenia show decreased NA, lower MI and elevated GLX when metabolites in all regions were averages. Regionally, schizophrenic subjects showed lower NA in frontal and temporal white matter regions, lower MI but higher GLX, and mildly increased CHO in most regions, but not all regions show statistical significance. Error bars indicate ±1 standard deviation.

Figure 3.

Figure 3

Open in a new tab

Group differences were observed for NA, MI and GLX across subjects groups. Relative to schizophrenic participants without cognitive decline (CDR <1), those with cognitive decline (CDR≥1) show greater decreases in NA in all regions, although the difference in the occipital region did not reached significance. However, both schizophrenic subgroups show variable decreases in MI and elevated GLX in these brain regions.

Figure 4.

Figure 4

Open in a new tab

Top and middle graphs: Right frontal white matter shows age-dependent decreases in CHO and total creatine in elderly schizophrenics (red triangles), which contrast with relatively stable CHO or trends for age-related increase in CR in the comparison participants (open circles). Similar trends for age-related changes in CHO and CR, but no group difference, were observed in the left frontal white matter. Bottom graphs: NA is lower in schizophrenia across the age span relative to the comparison participants. a = age-related decrease in CHO in schizophrenic subjects F(1,20)=11.6; r= −0.62; p=0.0029; b = group difference in age-related change in CHO: F(1,34)=4.54; p=0.04; c = age-related decrease in CR in schizophrenic subjects r= −0.62; p=0.0037; d = group difference in age-related change in CR: F(1,34)=5.84; p=0.02.

Results

Clinical Characteristics

Table 1 shows similar ages but lower education in the schizophrenic participants than our comparison participants. The two groups also showed similar hematocrit, thyroid stimulating hormones, and triglycerides, but the schizophrenia group had lower serum cholesterol. Only three comparison participants smoked cigarettes, while 15 of the 23 schizophrenic subjects were smokers. General cognitive function, as measured by Mini-Mental Status examination (MMSE), was also lower in the schizophrenia group. As expected, the schizophrenia group showed significantly higher CDR (15) than the comparison participants. The average age of onset of schizophrenia disorder was 21±2.4 years and duration of illness was 43.1±5.4 years. Participants with schizophrenia demonstrated a moderate amount of positive (mean PANSS positive score = 16.5±5.9) and negative (mean PANSS Negative score = 20±5.5) symptoms by the Positive And Negative Syndrome Scale (23).

Table 1.

Clinical Characteristics of the Participants

Schizophrenic Participants (n=23) Comparison Participants (n=22) t- and p- values
Age (years) 66.3±7.2 70.0±5.3 n.s.
Education (years) 12.2±1.8 15.5±3.6 t=4.0; df=43; p=0.0005
Hematocrit (%) 38.5±7.4 41.5±3.8 n.s.
TSH 2.57±1.9 1.98±1.1 n.s.
Triglycerides 115.4±62.9 118.1±38.7 n.s.
Cholesterol (mg/dL) 170.3±34.6 205.7±36.9 t=3.0; df=39; p=0.004
MMSE (0–30) 24.6±3.8 29.4±0.9 t=5.7; df=43; p<0.0001
CDR (stage 0–3) 1.13±0.97 0±0 t=5.5; df=43; p<0.0001
Open in a new tab

TSH = thyroid stimulating hormone; MMSE = Mini-Mental Status examination; CDR = Clinical Dementia Rating; n.s. = not significant.

All except two schizophrenic patients were treated with maintenance antipsychotic medications and some required additional psychotropic medications. Seven received both typical antipsychotics (5 haloperidol, 2 fluphenazine) and atypical antipsychotics (2 were treated with olazapine, 4 received risperidone and 1 received quetiapine concurrently). Of the remainder 16 patients, 14 were prescribed only atypical antipsychotics (5 received risperdone, 4 received quetiapine, 3 received olanzapine; 2 received clozapine), and 2 were not receiving antipsychotics at the time of the study. Ten patients required an anxiolytic medication (7 with lorazepam, 2 with clonazepam and 1 with buspirone). Nine participants also were receiving divalproex; three of these participants additionally were taking paroxetine, oxcarbamazepine or lithium.

Metabolite abnormalities in elderly participants with schizophrenia

Figure 2 show that the average metabolite level across all regions were different between the controls and the schizophrenic subjects for NA (−12.6%, p=0.0008), MI (−16.4%, p=0.026), and GLX (+28.7%, p=0.0016). These findings remain significant after correction for multiple comparisons using ranked-ordering of the significant values (22). The schizophrenic subjects showed only a trend for elevated CHO (+5.9%, p=0.18); no group difference was observed for CR. The individual regions showed group differences that were consistent with the overall findings, although not all regions reached statistical significance (Figure 2).

Potential influence of medications or drugs on brain metabolites

Since prior studies found lower NA only in schizophrenic patients who were receiving typical antipsychotics (24), we evaluated the potential differences in NA between typical and atypical antipsychotics. Both subgroups of schizophrenics showed lower NA across regions compared to controls (typical+atypical vs. controls: −10.2%, t29=2.3, p=0.03; atypical vs. controls: −17%, t32=3.6, p=0.001), but no group difference was observed between those who are treated with atypical versus typical+atypical antipspychotics.

We also evaluated the possible effect of valproic acid on glutamate levels (25); however, no group difference was observed in brain GLX levels in any brain regions in subjects with or without divaproex in their medication regimens. Furthermore, because nicotine-smoking might affect brain metabolites (26), we also compared brain metabolite levels in smoking and non-smoking schizophrenic participants, but found no group differences.

Metabolite abnormalities in elderly patients with and without cognitive decline

For the three metabolites that were different between schizophrenic subjects and controls, group differences were also found between schizophrenic subjects with definite cognitive decline (CDR≥1) and those without (CDR<1) and comparison subjects across regions (Figure 3). On post hoc analyses, schizophrenic subjects with cognitive decline showed significantly lower NA in all except the occipital region, while those without cognitive decline had significantly lower NA only in the frontal white matter regions. The elevated GLX showed a similar pattern of increase between the two schizophrenic subgroups. The decrease in MI was variable across regions between the subgroups.

Interaction between age and schizophrenia on brain metabolites

The right frontal white matter CHO and CR showed age-dependent decreases in the schizophrenic subjects, which contrasts with age-dependent increase in CR and relatively stable CHO levels in the right frontal white matter of non-schizophrenic participants (group difference in age-related change in CHO: F(1,34)=4.54; p=0.04; CR: F(1,34)=5.84; p=0.02; Figure 4, top left and middle left graphs). Similar trends for opposite slopes for age-related changes in CR were observed in the left frontal white matter; however, this difference in the slopes was not significant. Although NA is lower in the schizophrenic participants compared to healthy individuals across the age ranged studied, no group interaction related to age was observed (Figure 4, bottom graphs).

Discussion

Brain white matter metabolite abnormalities are shown in elderly patients with schizophrenia; specifically, we observed overall decreased NA and MI, elevated GLX, and a trend for elevation of CHO in schizophrenic subjects compared to healthy volunteers. Decreased NA suggests decreased neuronal content or neuronal dysfunction, while decreased MI suggests decreased glial content or function, and elevated GLX may reflect a hyperglutamatergic state in the white matter regions of these patients.

The decreases in the frontal and temporal white matter NA (12–18%) in these elderly schizophrenic patients are similar to or greater than those typically observed in the frontal gray matter (6.4% on average across 27 studies) or frontal white matter (6.4% matter across 17 studies) (13). The longer disease duration in our elderly schizophrenics might have contributed to the greater decreases in NA, as reported previously (24). Although neuropathology of schizophrenic brains consistently found increased, but maldistributed, neuronal density in the white matter (27), many also found decreased soma size (28), decreased dendritic arborization (29, 30), and hence fewer synaptic contacts and decreased brain volumes (31). Therefore, the decreased NA in the white matter likely reflects decreased neuronal content associated with decreased synapses and neuronal cell volumes (including the axons that traverse the white matter).

Imaging and neurocytochemical studies also showed white matter abnormalities in schizophrenic patients. For instance, MRI demonstrated decreased global and regional white matter volumes (32–35), decreased magnetization transfer ratios (36, 37), as well as decreased anisotropy on diffusion tensor imaging (DTI) (38–42) in various white matter brain regions of schizophrenic patients, all of which point to possible myelin or white matter abnormalities. Unlike acute inflammatory demyelinating disorders, such as multiple sclerosis (43) or active progressive multifocal leukoencephalopathy (PML) (44), we only observed a trend for increased CHO in the white matter of our elderly schizophrenic patients. The lack of significance might be due to the small effect size relative to the sample size.

However, unlike many acute inflammatory demyelinating brain disorders that show concomitant elevation of MI and CHO, our schizophrenic participants showed decreased MI in most brain regions studied. Since MI is a glial marker (45), it is typically elevated during glial activation or glial hypertrophy associated with active inflammation or demyelination, such as in the white matter of patients with HIV (46), multiple sclerosis (43, 47), or PML (44). The lack of elevated MI in patients with schizophrenia is also different from degenerative brain disorders, such as Alzheimer’s and frontotemporal dementia, which show increased MI along with decreased NA (48, 49). In these degenerative dementias, elevated MI probably reflects a glial response to the neuronal injury. Conversely, the decreased MI in our schizophrenic patients suggests decreased or dysfunctional glial response despite abnormal neuronal function (decreased NA). The decreased glial marker MI is also consistent with recent postmortem finding of reduced glial fibrillary acidic protein-reactive astroglia in the dorsal lateral prefrontal cortex of schizophrenic patients (50).

In addition, our schizophrenia subjects had elevations of white matter GLX. This finding is consistent with two previous studies that found elevated glutamate or GLX/CR in the frontal white matter of both medication-naïve and treated schizophrenic patients (12, 51). In contrast, another MRS study reported lower glutamate and glutamine levels in the anterior cingulate, but increased glutamine in the thalamus, of medicated schizophrenia patients (52). We found an average of 25.8% elevated GLX across brain regions, including the occipital regions, which tended to show normal levels for the other metabolites. The elevated GLX signal may reflect elevated glutamate, which could be due to a hyper-glutamatergic state of white matter in schizophrenia. Alternately, increased GLX may reflect a compensatory response to reduced glutamatergic transmission or binding to the receptors. Both possibilities, however, imply a pronounced dysfunction in the glutamate-glutamine homeostasis.

Since one important function of glia, especially astroglia, is re-uptake of glutamate from the extracellular space after neurotransmission, astroglial dysfunction might prevent this reuptake and lead to excess extracellular glutamate. Therefore, excess neuronal glutamate release coupled with dysfunctional glial reuptake could lead to the elevated extracelluar glutamate concentration. Among glial cells, oligodendroglia are particularly vulnerable to glutamate-mediated glial cell damage, via overactivation of AMPA (alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid) and kainate receptors (53). Interestingly, patients with multiple sclerosis also showed elevated glutamate on MRS in acute demyelinating lesions (43). Since multiple sclerosis is associated with pathology in the oligodendroglia, and hence demyelination, similar mechanism of astroglial dysfunction may contribute to the disease.

One limitation of the current study is that with a standard short-echo time MRS sequence (TE=30 ms), the spectral peaks of glutamate, glutamine and GABA were overlapping and difficult to separate even on the 4 Tesla scanner. Therefore, while GLX was fitted only from glutamate and glutamine concentrations, it is possible that some of the GABA signal was attributed to “GLX”, despite the separate fitting of GABA in the basis function. However, the contribution of GABA to the GLX would be minimal, since typical GABA concentrations are substantially (10–20 times) lower than those in the GLX peak. In addition, macromolecules might contribute to the GLX peak, as observed from degradation products in acute multiple sclerosis lesions (54). However, our LC Model analyses indicated little or no effect on the GLX levels from the macromolecule resonances. A novel technique, echo-time (TE)-averaged PRESS, can measure glutamate with minimal contamination of the glutamine and macromolecule peaks (55) which may be useful for future studies of schizophrenic patients. A second potential confound may be the effect of subtle differences in the partial volume of gray matter in the voxels between participant groups on metabolite concentrations. However, the relatively large changes in metabolite concentrations measured are much larger than the small concentration difference due to the different proportion of gray-white composition within the voxels. Thirdly, the potential effects of chronic medications on brain metabolite levels should be considered. Specifically, valproic acid and lithium both can stimulate glutamate release; however, the 9 schizophrenic participants who received divalproex and one that additionally received lithium did not show group difference in GLX any of the brain regions. Our finding is consistent with a recent MRS study that valproic acid had no effect on GLX in patients with bipolar disorder (56). Lastly, since chronic D2 blockade may lead to both structural and metabolic changes in the brains of schizophrenic patients (57, 58), decreased NA also may be related to the effects of antipsychotics. A recent study in rats treated long-term with haloperidol, however, showed no effect on brain NA or glutamate levels (59). Therefore, the metabolite changes observed in the current study are probably not due to medication effects; however, a larger sample size is needed to further validate these findings.

Schizophrenic subjects with cognitive decline had greater decreases in white matter NA than those without cognitive decline. In contrast to the greater than normal age-related decline of NA in younger participants with schizophrenia (24), elderly schizophrenic patients showed a similar slope but lower NA across the age span compared to controls. The prior study (24), however, did not correct for the partial volume effect from CSF in the voxels, which may be larger due to greater brain atrophy in the aging schizophrenic subjects. Postmortem studies indeed suggest schizophrenia does not involve ongoing neuronal degeneration (28). Although glutamatergic dysfunction is thought to be related to cognitive impairment in schizophrenia, both subjects with or without cognitive decline showed elevated GLX. Further correlation with cognitive testing and longitudinal studies are needed to assess the relationship between glutamate and cognitive function.

Assuming that age-related decline in CHO is associated with loss of myelin, the age by disease status interaction effects on CHO in the frontal white matter would support the hypothesis that elderly patients with schizophrenia may develop greater age-related myelin loss. CR also decreased with age in the schizophrenic subjects, which would suggest lower energy requirement and lack of age-related glial proliferation in the normal aging process, and age-associated elevation of CR and MI (60–62). These changes could lead to slower neuronal conduction and decline in frontal lobe function (e.g. executive function or working memory), as had been reported in elderly patients with schizophrenia (63, 64). Since glutamate reuptake is also important for shaping of excitatory post-synaptic currents (65), dysfunctional or decreased glutamate reuptake by glia may also interfere with neuronal function. Further evaluation of the relationships between MRS abnormalities to cognitive function in these patients is needed.

Acknowledgments

We are grateful to the research subjects who participated in this study. We also thank Dr. Michael Sanders for his participation in subject evaluation, and Drs. Dardo Tomasi and Elisabeth Caparelli for some of the MRS data acquisition and Renat Yakupov for technical assistance.

Financial Disclosure: This work was supported in part by funds from NIH [P50 MH 66392, a Conte Center Grant, awarded to Kenneth Davis; K24-DA16170 to LC; K02-DA16991 to TE], VISN3 MIRECC, and Department of Energy (Office of Biological and Environmental Research). None of the authors have any biomedical financial interests or potential conflicts of interest.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Davis K, Stewart D, Friedman J, Buchsbaum M, Harvey P, Hof P, et al. White matter changes in schizophrenia: evidence for myelin-related dysfunction. Archives of General Psychiatry. 2003;60:443–456. doi: 10.1001/archpsyc.60.5.443. [DOI] [PubMed] [Google Scholar]
  • 2.Torrey E, Webster M, Knable M, Johnston N, Yolken R. The stanley foundation brain collection and neuropathology consortium. Schizophrenia Research. 2000;44:151–155. doi: 10.1016/S0920-9964(99)00192-9. [DOI] [PubMed] [Google Scholar]
  • 3.Uranova N, Vostrikov V, Orlovskaya D, Rachmanova V. Oligodendroglial density in the prefrontal cortex in schizophrenia and mood disorders: a study from the Stanley Neuropathology Consortium. Schizophrenia Research. 2004;67:269–275. doi: 10.1016/S0920-9964(03)00181-6. [DOI] [PubMed] [Google Scholar]
  • 4.Hakak Y, Walker J, Li C, Wong W, Davis K, Buxbaum J, et al. Genome-wide expression analysis reveals dysregulation of myelination-related genes in chronic schizophrenia. Proceedings of the National Academy of Sciences U S A. 2001;98:4746–4751. doi: 10.1073/pnas.081071198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Peters A, Sethares C. Aging and the myelinated fibers in prefrontal cortex and corpus callosum of the monkey. Journal of Comparative Neurology. 2002;442:277–291. doi: 10.1002/cne.10099. [DOI] [PubMed] [Google Scholar]
  • 6.Charlton RA, Barrick TR, McIntyre DJ, Shen Y, O’Sullivan M, Howe FA, et al. White matter damage on diffusion tensor imaging correlates with age-related cognitive decline. Neurology. 2006;66:217–222. doi: 10.1212/01.wnl.0000194256.15247.83. [DOI] [PubMed] [Google Scholar]
  • 7.Javitt DC. Glutamate and Schizophrenia: Phencyclidine, N-Methyl-d-Aspartate Receptors, and Dopamine-Glutamate Interactions. International Review Neurobiology. 2007;78:69–108. doi: 10.1016/S0074-7742(06)78003-5. [DOI] [PubMed] [Google Scholar]
  • 8.Stone JM, Morrison P, Pilowsky LS. Glutamate and dopamine dysregulation in schizophrenia - a synthesis and selective review. Journal of Psychopharmacology. 2007 doi: 10.1177/0269881106073126. [DOI] [PubMed] [Google Scholar]
  • 9.Coyle JT. Glutamate and schizophrenia: beyond the dopamine hypothesis. Cellular Molecular Neurobiology. 2006;26:365–384. doi: 10.1007/s10571-006-9062-8. [DOI] [PubMed] [Google Scholar]
  • 10.Carter CJ. Schizophrenia susceptibility genes converge on interlinked pathways related to glutamatergic transmission and long-term potentiation, oxidative stress and oligodendrocyte viability. Schizophrenia Research. 2006;86:1–14. doi: 10.1016/j.schres.2006.05.023. [DOI] [PubMed] [Google Scholar]
  • 11.Theberge J, Bartha R, Drost D, Menon R, Malla A, Takhar J, et al. Glutamate and glutamine measured with 4.0 T proton MRS in never-treated patients with schizophrenia and healthy volunteers. American Journal of Psychiatry. 2002;159:1944–1946. doi: 10.1176/appi.ajp.159.11.1944. [DOI] [PubMed] [Google Scholar]
  • 12.Tebartz van Elst L, Valerius G, Buchert M, Thiel T, Rusch N, Bubl E, et al. Increased prefrontal and hippocampal glutamate concentration in schizophrenia: evidence from a magnetic resonance spectroscopy study. Biological Psychiatry. 2005;58:724–730. doi: 10.1016/j.biopsych.2005.04.041. [DOI] [PubMed] [Google Scholar]
  • 13.Steen RG, Hamer RM, Lieberman JA. Measurement of brain metabolites by 1H magnetic resonance spectroscopy in patients with schizophrenia: a systematic review and meta-analysis. Neuropsychopharmacology. 2005;30:1949–1962. doi: 10.1038/sj.npp.1300850. [DOI] [PubMed] [Google Scholar]
  • 14.Andreasen N, Flaum M, Arndt S. The comprehensive assessment of symptoms and history (CASH): An instrument for assessing psychopathology and diagnosis. Archives of General Psychiatry. 1992;49:615–623. doi: 10.1001/archpsyc.1992.01820080023004. [DOI] [PubMed] [Google Scholar]
  • 15.Berg L. Clinical dementia rating (CDR) Psychopharmacology Bulletin. 1988;24:637–639. [PubMed] [Google Scholar]
  • 16.Lee JH, Garwood M, Menon R, Adriany G, Andersen P, Truwit CL, et al. High contrast and fast three-dimensional magnetic resonance imaging at high fields. Magnetic Resonance in Medicine. 1995;34:308–312. doi: 10.1002/mrm.1910340305. [DOI] [PubMed] [Google Scholar]
  • 17.Hennig J, Kluge T, Scheffler K. Multiecho sequences with variable refocusing flip angles: Optimization of signal behavior using smooth transitions between pseudo steady states (TRAPS). Annual Meeting of the ISMRM; 2002; Honolulu. 2002. p. 2356. [DOI] [PubMed] [Google Scholar]
  • 18.Ernst T, Chang L. Elimination of Artifacts in Short Echo Time 1H MR Spectroscopy of the Frontal Lobe. Magnetic Resonance in Medicine. 1996;36:462–468. doi: 10.1002/mrm.1910360320. [DOI] [PubMed] [Google Scholar]
  • 19.Ernst T, Kreis R, Ross BD. Absolute quantitation of water and metabolites in the human brain. I: compartments and water. Journal of Magnetic Resonance. 1993;B102:1–8. [Google Scholar]
  • 20.Kreis R, Ernst T, Ross BD. Absolute quantitation of water and metabolites in the human brain. II: metabolite concentrations. Journal of Magnetic Resonance. 1993;B102:9–19. [Google Scholar]
  • 21.Provencher S. Estimation of metabolite concentrations from localized in vivo proton NMR spectra. Magnetic Resonance in Medicine. 1993;30:672. doi: 10.1002/mrm.1910300604. [DOI] [PubMed] [Google Scholar]
  • 22.Simes RJ. An improved Bonferroni procedure for multiple tests of significance. Biometrika. 1986;73:751–754. [Google Scholar]
  • 23.Kay S. Positive and negative syndromes in schizophrenia. New York: Brunner/Mazel; 1991. [Google Scholar]
  • 24.Ende G, Dieter B, Walter S, Weber-Fahr W, Soher B, Maudsley A, et al. Effects of age, medication, and illness duration on the N-Acetylaspartate signal of the anterior cingulate region in schizophrenia. Schizophrenia Research. 2000;41:389–395. doi: 10.1016/s0920-9964(99)00089-4. [DOI] [PubMed] [Google Scholar]
  • 25.Dixon JF, Hokin LE. The antibipolar drug valproate mimics lithium in stimulating glutamate release and inositol 1,4,5-trisphosphate accumulation in brain cortex slices but not accumulation of inositol monophosphates and bisphosphates. Proc Natl Acad Sci U S A. 1997;94:4757–4760. doi: 10.1073/pnas.94.9.4757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Durazzo TC, Gazdzinski S, Banys P, Meyerhoff DJ. Cigarette smoking exacerbates chronic alcohol-induced brain damage: a preliminary metabolite imaging study. Alcohol Clin Exp Res. 2004;28:1849–1860. doi: 10.1097/01.alc.0000148112.92525.ac. [DOI] [PubMed] [Google Scholar]
  • 27.Akbarian S, Kim J, Potkin S, Hetrick W, Bunney WJ, Jones E. Maldistribution of interstitial neurons in prefrontal white matter of the brains of schizophrenic patients. Archives of General Psychiatry. 1996;53:425–436. doi: 10.1001/archpsyc.1996.01830050061010. [DOI] [PubMed] [Google Scholar]
  • 28.Rajkowska G, Selemon L, Goldman-Rakic P. Neuronal and glial somal size in the prefrontal cortex: a postmortem morphometric study of schizophrenia and Huntington disease. Arch Gen Psychiatry. 1998;55:215–224. doi: 10.1001/archpsyc.55.3.215. [DOI] [PubMed] [Google Scholar]
  • 29.Brown R, Colter N, Corsellis J, Crow T, Frith C, Jagoe R, et al. Postmortem evidence of structural brain changes in schizophrenia. Differences in brain weight, temporal horn area, and parahippocampal gyrus compared with affective disorder. Archives of General Psychiatry. 1986;43:36–42. doi: 10.1001/archpsyc.1986.01800010038005. [DOI] [PubMed] [Google Scholar]
  • 30.Pakkenberg B. Post-mortem study of chronic schizophrenic brains. British Journal of Psychiatry. 1987;151:744–752. doi: 10.1192/bjp.151.6.744. [DOI] [PubMed] [Google Scholar]
  • 31.Andreasen N, Flashman L, Flaum M, Arndt S, Swayze Vn, O’Leary D, et al. Regional brain abnormalities in schizophrenia measured with magnetic resonance imaging. Journal of the American Medical Association. 1994;272:1763–1769. [PubMed] [Google Scholar]
  • 32.Okugawa G, Sedvall G, Agartz I. Reduced grey and white matter volumes in the temporal lobe of male patients with chronic schizophrenia. European Archives of Psychiatry and Clinical Neuroscience. 2002;252:120–123. doi: 10.1007/s00406-002-0370-9. [DOI] [PubMed] [Google Scholar]
  • 33.Takase K, Tamagaki C, Okugawa G, Nobuhara K, Minami T, Sugimoto T, et al. Reduced white matter volume of the caudate nucleus in patients with schizophrenia. Neuropsychobiology. 2004;50:296–300. doi: 10.1159/000080956. [DOI] [PubMed] [Google Scholar]
  • 34.Hulshoff Pol H, Brans R, van Haren N, Schnack H, Langen M, Baare W, et al. Gray and white matter volume abnormalities in monozygotic and same-gender dizygotic twins discordant for schizophrenia. Biological Psychiatry. 2004;55:126–130. doi: 10.1016/s0006-3223(03)00728-5. [DOI] [PubMed] [Google Scholar]
  • 35.Hulshoff Pol H, Schnack H, Mandl R, Cahn W, Collins D, Evans A, et al. Focal white matter density changes in schizophrenia: reduced inter-hemispheric connectivity. Neuroimage. 2004;21:27–35. doi: 10.1016/j.neuroimage.2003.09.026. [DOI] [PubMed] [Google Scholar]
  • 36.Foong J, Symms M, Barker G, Maier M, Woermann F, Miller D, et al. Neuropathological abnormalities in schizophrenia: evidence from magnetization transfer imaging. Brain. 2001;124:882–892. doi: 10.1093/brain/124.5.882. [DOI] [PubMed] [Google Scholar]
  • 37.Bagary M, Symms M, Barker G, Mutsatsa S, Joyce E, Ron M. Gray and white matter brain abnormalities in first-episode schizophrenia inferred from magnetization transfer imaging. Archives of General Psychiatry. 2003;60:779–788. doi: 10.1001/archpsyc.60.8.779. [DOI] [PubMed] [Google Scholar]
  • 38.Buchsbaum M, Tang C, Peled S, Gudbjartsson H, Lu D, Hazlett E, et al. MRI white matter diffusion anisotropy and PET metabolic rate in schizophrenia. Neuroreport. 1998;9:425–430. doi: 10.1097/00001756-199802160-00013. [DOI] [PubMed] [Google Scholar]
  • 39.Lim K, Hedehus M, Moseley M, de Crespigny A, Sullivan E, Pfefferbaum A. Compromised white matter tract integrity in schizophrenia inferred from diffusion tensor imaging. Archives of General Psychiatry. 1999;56:367–394. doi: 10.1001/archpsyc.56.4.367. [DOI] [PubMed] [Google Scholar]
  • 40.Agartz I, Andersson J, Skare S. Abnormal brain white matter in schizophrenia: a diffusion tensor imaging study. Neuroreport. 2001;12:2251–2254. doi: 10.1097/00001756-200107200-00041. [DOI] [PubMed] [Google Scholar]
  • 41.Wolkin A, Choi S, Szilagyi S, Sanfilipo M, Rotrosen J, Lim K. Inferior frontal white matter anisotropy and negative symptoms of schizophrenia: a diffusion tensor imaging study. American Journal of Psychiatry. 2003;160:572–574. doi: 10.1176/appi.ajp.160.3.572. [DOI] [PubMed] [Google Scholar]
  • 42.Szeszko P, Ardekani B, Ashtari M, Kumra S, Robinson D, Sevy S, et al. White matter abnormalities in first-episode schizophrenia or schizoaffective disorder: a diffusion tensor imaging study. American Journal of Psychiatry. 2005;162:602–605. doi: 10.1176/appi.ajp.162.3.602. [DOI] [PubMed] [Google Scholar]
  • 43.Srinivasan R, Sailasuta N, Hurd R, Nelson S, Pelletier D. Evidence of elevated glutamate in multiple sclerosis using magnetic resonance spectroscopy at 3 T. Brain. 2005;128:1016–1025. doi: 10.1093/brain/awh467. [DOI] [PubMed] [Google Scholar]
  • 44.Chang L, Ernst T, Tornatore C, Aronow H, Melchor R, Walot I, et al. Metabolite abnormalities in progressive multifocal leukoencephalopathy by proton magnetic resonance spectroscopy. Neurology. 1997;48:836–845. doi: 10.1212/wnl.48.4.836. [DOI] [PubMed] [Google Scholar]
  • 45.Brand A, Richter-Landsberg C, Leibfritz D. Multinuclear NMR studies on the energy metabolism of glial and neuronal cells. Developmental Neurosciences. 1993;15:289–298. doi: 10.1159/000111347. [DOI] [PubMed] [Google Scholar]
  • 46.Chang L, Ernst T, Leonido-Yee M, Walot I, Singer E. Cerebral metabolite abnormalities correlate with clinical severity of HIV-cognitive motor complex. Neurology. 1999;52:100–108. doi: 10.1212/wnl.52.1.100. [DOI] [PubMed] [Google Scholar]
  • 47.Fernando K, McLean M, Chard D, MacManus D, Dalton C, Miszkiel K, et al. Elevated white matter myo-inositol in clinically isolated syndromes suggestive of multiple sclerosis. Brain. 2004 Jun;:127. doi: 10.1093/brain/awh153. Epub 2004 May 5, 127:1361–1369. [DOI] [PubMed] [Google Scholar]
  • 48.Ernst T, Chang L, Melchor R, Mehringer CM. Frontotemporal Dementia and Early Alzheimer Disease: Differentiation with Frontal Lobe H-1 MR Spectroscopy. Radiology. 1997;203:829–836. doi: 10.1148/radiology.203.3.9169712. [DOI] [PubMed] [Google Scholar]
  • 49.Huang W, Alexander G, Chang L, Shetty H, Krasuski J, Rapoport S, et al. Brain metabolite concentration and dementia severity in Alzheimer’s disease: a (1)H MRS study. Neurology. 2001;57:626–632. doi: 10.1212/wnl.57.4.626. [DOI] [PubMed] [Google Scholar]
  • 50.Rajkowska G, Miguel-Hidalgo J, Makkos Z, Meltzer H, Overholser J, Stockmeier C. Layer-specific reductions in GFAP-reactive astroglia in the dorsolateral prefrontal cortex in schizophrenia. Schizophr Res. 2002 Oct 1;57(2–3):127–138. doi: 10.1016/s0920-9964(02)00339-0. [DOI] [PubMed] [Google Scholar]
  • 51.Choe BY, Kim KT, Suh TS, Lee C, Paik IH, Bahk YW, et al. 1H magnetic resonance spectroscopy characterization of neuronal dysfunction in drug-naive, chronic schizophrenia. Academic Radiology. 1994;1:211–216. doi: 10.1016/s1076-6332(05)80716-0. [DOI] [PubMed] [Google Scholar]
  • 52.Theberge J, Al-Semaan Y, Williamson P, Menon R, Neufeld R, Rajakumar N, et al. Glutamate and glutamine in the anterior cingulate and thalamus of medicated patients with chronic schizophrenia and healthy comparison subjects measured with 4.0-T proton MRS. American Journal of Psychiatry. 2003;160:2231–2233. doi: 10.1176/appi.ajp.160.12.2231. [DOI] [PubMed] [Google Scholar]
  • 53.Matute C, Domercq M, Sanchez-Gomez MV. Glutamate-mediated glial injury: mechanisms and clinical importance. Glia. 2006;53:212–224. doi: 10.1002/glia.20275. [DOI] [PubMed] [Google Scholar]
  • 54.Mader I, Seeger U, Weissert R, Klose U, Naegele T, Melms A, et al. Proton MR spectroscopy with metabolite-nulling reveals elevated macromolecules in acute multiple sclerosis. Brain. 2001;124:953–961. doi: 10.1093/brain/124.5.953. [DOI] [PubMed] [Google Scholar]
  • 55.Hurd R, Sailasuta N, Srinivasan R, Vigneron DB, Pelletier D, Nelson SJ. Measurement of brain glutamate using TE-averaged PRESS at 3T. Magnetic Resonance in Medicine. 2004;51:435–440. doi: 10.1002/mrm.20007. [DOI] [PubMed] [Google Scholar]
  • 56.Friedman SD, Dager SR, Parow A, Hirashima F, Demopulos C, Stoll AL, et al. Lithium and valproic acid treatment effects on brain chemistry in bipolar disorder. Biol Psychiatry. 2004;56:340–348. doi: 10.1016/j.biopsych.2004.06.012. [DOI] [PubMed] [Google Scholar]
  • 57.Corson PW, Nopoulos P, Miller DD, Arndt S, Andreasen NC. Change in basal ganglia volume over 2 years in patients with schizophrenia: typical versus atypical neuroleptics. Am J Psychiatry. 1999;156:1200–1204. doi: 10.1176/ajp.156.8.1200. [DOI] [PubMed] [Google Scholar]
  • 58.Corson PW, O’Leary DS, Miller del D, Andreasen NC. The effects of neuroleptic medications on basal ganglia blood flow in schizophreniform disorders: a comparison between the neuroleptic-naive and medicated states. Biol Psychiatry. 2002;52:855–862. doi: 10.1016/s0006-3223(02)01421-x. [DOI] [PubMed] [Google Scholar]
  • 59.Bustillo J, Barrow R, Paz R, Tang J, Seraji-Bozorgzad N, Moore GJ, et al. Long-term treatment of rats with haloperidol: lack of an effect on brain N-acetyl aspartate levels. Neuropsychopharmacology. 2006;31:751–756. doi: 10.1038/sj.npp.1300874. [DOI] [PubMed] [Google Scholar]
  • 60.Chang L, Ernst T, Poland R, Jenden D. In vivo proton magnetic resonance spectroscopy of the normal human aging brain. Life Sciences. 1996;58:2049–2056. doi: 10.1016/0024-3205(96)00197-x. [DOI] [PubMed] [Google Scholar]
  • 61.Pfefferbaum A, Adalsteinsson E, Spielman D, Sullivan EV, Lim KO. In vivo spectroscopic quantification of the N-acetyl moiety, creatine, and choline from large volumes of brain gray and white matter: effects of normal aging. Magnetic Resonance in Medicine. 1999;41:276–284. doi: 10.1002/(sici)1522-2594(199902)41:2<276::aid-mrm10>3.0.co;2-8. [DOI] [PubMed] [Google Scholar]
  • 62.Schuff N, Ezekiel F, Gamst AC, Amend DL, Capizzano AA, Maudsley AA, et al. Region and tissue differences of metabolites in normally aged brain using multislice 1H magnetic resonance spectroscopic imaging. Magnetic Resonance in Medicine. 2001;45:899–907. doi: 10.1002/mrm.1119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Morey R, Inan S, Mitchell T, Perkins D, Lieberman J, Belger A. Imaging frontostriatal function in ultra-high-risk, early, and chronic schizophrenia during executive processing. Archives of General Psychiatry. 2005;62:254–262. doi: 10.1001/archpsyc.62.3.254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Krieger S, Lis S, Cetin T, Gallhofer B, Meyer-Lindenberg A. Executive function and cognitive subprocesses in first-episode, drug-naive schizophrenia: an analysis of N-back performance. American Journal of Psychiatry. 2005;162:1206–1208. doi: 10.1176/appi.ajp.162.6.1206. [DOI] [PubMed] [Google Scholar]
  • 65.Auger C, Attwell D. Fast removal of synaptic glutamate by postsynaptic transporters. Neuron. 2000;28:547–558. doi: 10.1016/s0896-6273(00)00132-x. [DOI] [PubMed] [Google Scholar]

RESOURCES