Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Oct 1.
Published in final edited form as: Brain Imaging Behav. 2020 Oct;14(5):1419–1429. doi: 10.1007/s11682-019-00060-4

A Magnetic Resonance Spectroscopy Investigation in Symptomatic Former NFL Players

Michael L Alosco 1, Yorghos Tripodis 2, Benjamin Rowland 3, Alicia S Chua 4, Huijun Liao 5, Brett Martin 6, Johnny Jarnagin 7, Christine E Chaisson 8, Ofer Pasternak 9, Sarina Karmacharya 10, Inga K Koerte 11, Robert C Cantu 12, Neil W Kowall 13, Ann C McKee 14, Martha E Shenton 15, Richard Greenwald 16, Michael McClean 17, Robert A Stern 18,*, Alexander Lin 19,*
PMCID: PMC6994233  NIHMSID: NIHMS1069210  PMID: 30848432

Abstract

The long-term neurologic consequences of exposure to repetitive head impacts (RHI) are not well understood. This study used magnetic resonance spectroscopy (MRS) to examine later-life neurochemistry and its association with RHI and clinical function in former National Football League (NFL) players. The sample included 77 symptomatic former NFL players and 23 asymptomatic individuals without a head trauma history. Participants completed cognitive, behavior, and mood measures. N-acetyl aspartate, glutamate/glutamine, choline, myo-inositol, creatine, and glutathione were measured in the posterior (PCG) and anterior (ACG) cingulate gyrus, and parietal white matter (PWM). A cumulative head impact index (CHII) estimated RHI. In former NFL players, a higher CHII correlated with lower PWM creatine (r=−0.23, p=0.02). Multivariate mixed-effect models examined neurochemical differences between the former NFL players and asymptomatic individuals without a history of head trauma. PWM N-acetyl aspartate was lower among the former NFL players (mean diff.=1.02, p=0.03). Between-group analyses are preliminary as groups were recruited based on symptomatic status. The ACG was the only region associated with clinical function, including positive correlations between glutamate (r=0.32, p=0.004), glutathione (r=0.29, p=0.02), and myo-inositol (r=0.26, p=0.01) with behavioral/mood symptoms. Other positive correlations between ACG neurochemistry and clinical function emerged (i.e., behavioral/mood symptoms, cognition), but the positive directionality was unexpected. All analyses controlled for age, body mass index, and education (for analyses examining clinical function). In this sample of symptomatic former NFL players, there was a direct effect between RHI and reduced cellular energy metabolism (i.e., lower creatine). MRS neurochemicals associated with neuroinflammation also correlated with behavioral/mood symptoms.

Keywords: Chronic traumatic encephalopathy, repetitive head impacts, magnetic resonance spectroscopy, magnetic resonance imaging, tackle football

INTRODUCTION

Chronic traumatic encephalopathy (CTE) is a neurodegenerative disease found in individuals exposed to repetitive head impacts (RHI), particularly American football players (McKee et al. 2013; Mez et al. 2017). The neuropathology of CTE is well-defined (McKee et al. 2016; McKee et al. 2013; Mez et al. 2017). Our understanding on its clinical presentation has improved (Alosco, Mez, et al. 2018; Mez et al. 2017; Montenigro et al. 2014; Stern et al. 2013), but CTE cannot be diagnosed in life because validated biomarkers do not exist. Proton MR spectroscopy (MRS) may assist in the detection of CTE. MRS measures neurochemical markers of neuronal viability (N-acetyl aspartate; NAA), immunoexcitotoxicity (glutamate [Glu]/glutamine; Glx), axonal injury (choline; Cho), astrocytosis and microglial activation (myo-inositol; mI), energy metabolism (creatine; Cr), and neuroinflammation (glutathione; GSH). CTE is associated with widespread neurodegeneration with the diagnostic lesion being perivascular phosphorylated tau (p-tau) at the sulcal depths (McKee et al. 2016). Neuroinflammation, gliosis, axonal loss, and white matter disease accompany CTE (McKee et al. 2013). MRS has indeed been used to detect neurodegenerative diseases, like Alzheimer’s disease (AD) (Graff-Radford & Kantarci 2013; Kantarci et al. 2007; Watanabe, Shiino, & Akiguchi 2012).

MRS studies in former contact sport athletes (e.g., former professional soccer players, boxers, rugby players) (Davie et al. 1995; Gardner et al. 2017; Koerte et al. 2015; Lin et al. 2015; Tremblay et al. 2013) link RHI with long-term neurochemical alterations, which may contribute to cognition (Koerte et al. 2015; Tremblay et al. 2013). The few studies examining RHI and later-life neurochemistry used small sample sizes and lacked robust metrics to estimate RHI. This study used MRS to examine neurochemical concentrations in symptomatic former National Football League (NFL) players. The relationship between RHI and neurochemistry was tested, as were the associations between neurochemistry and neuropsychological and neuropsychiatric functioning. We hypothesized that NAA and higher Glu, Glx, GSH, and Cho would correlate with RHI and be associated with worse cognitive and neuropsychiatric functioning.

MATERIALS AND METHODS

Participants

Participants were from the “Diagnosing and Evaluating Traumatic Encephalopathy using Clinical Tests” (DETECT) study. Participants included 96 former NFL players. Inclusion criteria included: male, aged 40–69, a minimum of two NFL seasons and a minimum of twelve years of organized football, and self-reported complaints of cognitive, and/or behavioral/mood symptoms at the time of study screening. A same-age comparison group (n=24), who were without a history of contact sport participation, service in the military, or self-reported TBI, and denied symptoms (at telephone screen) were recruited. Since the groups differed according to RHI history and symptom status, all analyses with this “control” group were considered preliminary. Exclusion criteria for all participants included MRI and/or lumbar puncture contraindications, presence of another central nervous system disease, primary language other than English, and/or history of a TBI within one year of study screening.

Participants completed a two- to three-day study visit that included: demographic, medical, and athletic history interview(s); neurological evaluation; neuropsychological testing; structured psychiatric interview; self-report behavior/mood measures; neuroimaging; among other exams not relevant to this study (Alosco, Jarnagin, et al. 2017; Alosco, Tripodis, et al. 2018; Stamm, Bourlas, et al. 2015; Stamm, Koerte, et al. 2015).

Standard Protocol Approvals, Registrations, and Patient Consents

Study protocols were approved by the Institutional Review Boards at Boston University Medical Campus and Brigham and Women’s Hospital and were HIPAA compliant. All participants provided written informed consent.

Measures

Magnetic Resonance Spectroscopy

The MR scanner was a 3T Siemens Verio with a 32-channel head coil array. The MRI protocol consisted of a 3D T1-weighted sequence (MPRAGE [1×1×1mm3, TR=1800ms, TE=3.36ms)]), and a 3D T2-weighted sequence (SPACE, [1×1×1mm3, TR=3200ms, TE=456ms]). Single voxel point-resolved spectroscopy (PRESS; TE=35ms, TR=2s, 2×2×2cm3) measured NAA, mI, Cr, Cho, Glu, Glx, and GSH concentrations in the anterior (ACG) and posterior cingulate gyrus (PCG), and the left parietal white matter (PWM) (Figure 1). The ACG, PCG, and PWM are sensitive to neurochemical changes in AD (Fayed, Modrego, Rojas-Salinas, & Aguilar 2011; Graff-Radford & Kantarci 2013) and provide optimal spectral quality. The voxel was localized using anatomical landmarks from T1-weighted images. Each voxel underwent automated optimization (3D shimming, transmit gain, frequency adjustment, and water suppression). Screen shots of voxel location and spectra were examined. Manual shimming to a line width of <14 Hz of the full-width half maximum (FWHM) of the unsuppressed water spectrum was performed.

Fig. 1.

Fig. 1

Representative Voxel Locations and Spectra. (A) Voxel location (sagittal and axial, respectively) and spectra for the anterior cingulate gyrus, (B) Voxel location (sagittal and axial, respectively) and spectra for the posterior cingulate gyrus, (C) Voxel location (sagittal and axial, respectively) and spectra for the parietal white matter. Former NFL players were required to be symptomatic and “Controls” were individuals who were asymptomatic and did not have a history of head trauma.

The single voxel MRS spectra were exported and raw data were processed by the study physicist (B.R.), who was blind to group membership, using singular value decomposition (SVD) based channel combination, spectral registration to correct for frequency drift, and residual water suppression using the Hankel SVD method (Rowland et al. 2017). The metabolites were fit using linear combination models (LCModel). Metabolites FWHM were <0.10 and signal to noise ratio was <50. To optimize data quality and reliability, only metabolites with Cramer-Rao lower bound (CRLB) <20% were analyzed; a majority had a CRLB <10%. Concentrations were partial-volume corrected by segmenting gray matter, white matter, and CSF within the voxel and correcting for the water concentration. To do so, the concentrations were multiplied by the following factor:

[(55556/35880)×(1.000×partialvolumefractionofCSF)+(0.779×partialvolumeofgraymatter)+(0.645×partialvolumeofwhitematter)]

This equation corrects for the assumed LCModel water concentration (35880mM) with the actual water concentration (55556mM), and then corrects for partial volume of each compartment of water, where CSF is assumed 100% water content, gray matter is 77.9%, and white matter is 64.5% (Ernst 1993).

Cumulative Head Impact Index

Exposure to RHI was estimated by the CHII (Montenigro et al. 2017). The CHII is based on self-reported football history (number of seasons, position[s] at each level played), and estimated head impact exposure frequencies from published studies using helmet-mounted accelerometers. The CHII was developed in former amateur football players because published helmet accelerometer studies at the professional level do not exist. Estimates of head impact frequencies from published college accelerometer studies were applied to the seasons and positions played in the NFL in the present sample. Higher CHII reflects greater exposure to RHI.

Clinical Measures

Neuropsychological tests evaluated attention, executive function, verbal and visual episodic memory, language, and visuospatial function. Semi-structured interviews and self-report measures of neuropsychiatric function (e.g., depression, apathy, aggression) were completed. All tests administered are described elsewhere (Alosco, Jarnagin, et al. 2017). Raw scores were transformed to standard scores using normative data calibrated for age, sex, and/or education. Principal component analysis resulted in composite scores for behavioral/mood, psychomotor speed/executive function, verbal memory, and visual memory domains (Alosco, Jarnagin, et al. 2017). Higher scores on the behavioral/mood composite are worse, whereas lower scores on the other composites are worse.

Sample Size

The sample of 96 former NFL players and 24 asymptomatic individuals without a history of head trauma was reduced to 77 and 23, respectively, following exclusion of participants who did not complete MRI or whose structural MRI and/or MRS data acquisition was of inadequate quality due to motion artifact. Two additional former NFL players were excluded due to neuropsychological evidence of intentional symptom exaggeration. Table 1 presents sample characteristics. There were no differences between the 77 former NFL players compared to those excluded in terms of age, education, or years of football, or clinical test scores (p’s>0.05). Of the 77 former NFL players, four former NFL players did not have MRS data for the ACG (n=73) due to early termination of the MRI protocol; of these four, one did not have PWM data (n=76). As mentioned above, only neurometabolites with a CRLB <20% were examined. This criterion affected the sample size for GSH (PCG: n=75 former NFL players, n=23 comparison group; ACG: n=66 former NFL players, n=21 comparison group; PWM: n=66 former NFL players, n=22 comparison group). Finally, for analyses examining the clinical factor composite scores as the outcome, the sample size of the former NFL players was decreased to 69 due to missing data on individual cognitive, behavior, and/or mood measures that make-up the composite scores; note that the sample size was further reduced and varied across neurochemicals and brain regions due to the above described exclusionary reasons. For the controls, the sample size of 23 was constant with the exception of PWM GSH (n=22) and ACG GSH (n=21).

Table 1.

Sample Characteristics

Symptomatic Former NFL Players (n = 77) Controls (n = 23) p-value
Age, mean (SD) years 55.96 (7.77) 57.22 (6.89) 0.49
Education, mean (SD) years 16.45 (1.02) 17.30 (2.14) 0.08
African American, n (%) 30 (39.0) 1 (4.3) 0.002
Duration of football play, mean (SD) years 17.97 (3.59) -- --
Years in the NFL, mean (SD) 7.76 (2.75) -- --
Cumulative Head Impact Index, mean (SD) 19,881.71 (6540.99) -- --
Primary Position Group, n (%)
 Offensive line 19 (24.7) -- --
 Running back 7 (9.1) -- --
 Tight end 4 (5.2) -- --
 Offensive skill 1 (1.3) -- --
 Defensive line 11 (14.3) -- --
 Linebacker 18 (23.4) -- --
 Defensive Back 17 (22.1) -- --
Body mass index, mean (SD) kg/m2 32.73 (4.89) 28.02 (3.90) <0.001

The “control” group was required to be asymptomatic and have no history of head trauma at the time of recruitment. Independent samples t-tests tested for statistically significant group differences on age, education, and body mass index, whereas Fisher’s Exact Test (due to small cell sizes) was used for race.

Statistical Analysis

In the former NFL players, partial correlations controlling for age and body mass index (BMI) examined associations among the ACG, PCG, and PWM neurochemicals with the CHII and the clinical composite scores (i.e., behavioral/mood, psychomotor speed/executive function, verbal memory, and visual memory). Analyses examining clinical function controlled for education, along with age and BMI. Multivariate linear mixed-effects models examined differences between the former NFL players and the controls in NAA, mI, Cho, Cr, Glu, Glx, and GSH concentrations in the ACG, PCG, and PWM. Correlated outcomes can artificially decrease p-values (Sainani 2010). The multivariate linear mixed-effects models were thus performed to reduce Type I error because they account for correlations between groups, outcomes from the same participant, and between the same test for metabolites and brain regions. The models were adjusted for age and BMI. Race was not included as a covariate because there is not a large enough sample of African American controls (n=1) to provide an accurate estimation on the differential effects of race on the neurochemicals between the former NFL players and controls. The analyses of group differences are considered to be preliminary due to differences between the groups in symptomatic status at the time of recruitment.

For all analyses, bootstrap analysis was performed on 500 replicates to further control for Type I error and increase statistical power. All results presented are from the bootstrap analyses that protect against Type I error inflation for each individual p-value. Significance level was an alpha of 0.05. SAS (SAS Institute Inc., v.9.4) was used to perform analyses.

RESULTS

Table 1 provides an overview of the sample characteristics. The principal component factor composite scores for the symptomatic former NFL players and controls are presented in Table 2.

Table 2.

Cognitive and Neuropsychiatric Test Performance

Former NFL Players
Controls
P-value
Principle Component Factor Scores, mean (SD)
Behavioral/Mood 0.35 (0.93) −0.92 (0.46) <0.001
Psychomotor Speed/Executive Function −0.03 (0.84) 0.27 (0.68) 0.75
Verbal Memory −0.08 (0.89) 0.35 (1.22) 0.05
Visual Memory 0.01 (0.90) 0.29 (0.74) 0.28

The “control” group was required to be asymptomatic and have no history of head trauma at the time of recruitment. Neuropsychological tests evaluated attention, executive function, verbal and visual episodic memory, language, and visuospatial function. Semi-structured interviews and self-report measures of neuropsychiatric function (e.g., depression, apathy, aggression) were completed. Raw scores were transformed to standard scores using normative data calibrated for age, sex, and/or education. Principal component analysis resulted in composite scores for behavioral/mood, psychomotor speed/executive function, verbal memory, and visual memory domains. The sample size of the former NFL players was decreased to 69 due to missing data on individual cognitive, behavior, and/or mood measures that make-up the composite scores. The sample size was further reduced and varied across the neurochemicals and brain regions due to other participant exclusionary criterions described in the text. A multivariate analysis of covariance (ANCOVA) controlling for age, body mass index, and years of education was performed to test for group differences on each clinical factor composite score.

Exposure to RHI and Neurochemical Concentrations

In the former NFL players, a higher CHII was associated with lower later-life concentrations of Cr in the PWM (r=−0.23, p=0.02) (Figure 2). The CHII was not associated with NAA, Glu, Cho, Cr, mI, or GSH in the PWM (ps>0.10). In the ACG and PCG, there were no significant correlations with all of the measured metabolites (ps>0.20).

Fig 2.

Fig 2

Greater Exposure to Repetitive Head Impacts is Associated with Lower Later-life Parietal White Matter Creatine Concentrations. The scatter plot is of the residuals between the cumulative head impact index (CHII) and Creatine (Cr) after controlling for age and body mass index. X-axis represent each participant’s CHII, with higher scores reflecting greater exposure to RHI. Y-axis values are Cr concentrations. Shaded region is the 95% confidence intervals. The results remain unchanged when the individual with very low Cr concentrations was removed from the analysis.

Symptomatic Former NFL Players versus Asymptomatic Controls: Group Differences

A summary of the linear mixed-effect models is provided in Table 3. The former NFL players exhibited significantly lower PWM NAA levels compared to asymptomatic individuals without a history of head trauma (mean difference=1.02, 95% CI = 0.08, 2.31, p=0.03). No significant differences between the former NFL players and controls were found for the other neurochemicals in the PWM, ACG, or PCG (p’s>0.10).

Table 3.

Summary of Linear Mixed-Effects Models Examining Differences in Neurochemical Concentrations Between the Former NFL Players and Controls

Neurometabolites ACG PCG PWM
Mean Diff 95% CI P Mean Diff 95% CI P Mean Diff 95% CI P
Creatine 0.16 −0.64, 0.70 0.66 0.04 −0.74, 0.60 0.93 0.31 −0.14, 1.09 0.28
Glutamate_Glutamine −0.95 −2.26, 0.12 0.10 −0.47 −1.64, 0.46 0.37 −0.03 −0.06, 1.17 0.90
Glutamate −0.33 −1.25, 0.44 0.43 −0.14 −1.04, 0.62 0.78 0.14 −0.53, 1.39 0.80
Choline 0.17 −0.10, 0.39 0.26 0.09 −0.12, 0.23 0.36 0.26 −0.09, 0.52 0.08
Glutathione −0.09 −0.32, 0.16 0.47 −0.04 −0.25, 0.15 0.65 0.11 −0.14, 0.39 0.38
Myo-inositol 0.24 −0.50, 0.95 0.56 0.32 −0.34, 0.91 0.32 0.12 −0.57, 0.81 0.81
N-acetyl aspartate −0.27 −1.55, 0.92 0.60 −0.52 −1.57, 0.35 0.26 1.02 0.08, 2.31 0.03

The “control” group was required to be asymptomatic and have no history of head trauma at the time of recruitment. Analyses adjusted for age and body mass index. Mean diff = mean difference in neurochemical concentration between former NFL players and controls (former NFL-control). Bootstrap analysis was performed on 500 replicates to further control for Type I error and increase statistical power. ACG = anterior cingulate gyrus; PCG = posterior cingulate gyrus; and PWM = left parietal lobe white matter.

Neurochemistry and Clinical Function

Partial correlations examining the relationship between neurochemistry in the PWM, ACG, and PCG with behavior/mood, psychomotor speed/executive function, and verbal and visual memory in the former NFL players are presented in Table 4. The ACG was the only brain region associated with clinical function. In particular, there were statistically significant positive correlations between ACG concentrations of Glu, GSH, and mI with the behavioral/mood composite score (Figure 3). Higher concentrations of NAA in the ACG were also associated with higher scores (i.e., worse) on the behavioral/mood composite. There was a statistically significant positive effect between ACG Glx concentrations and psychomotor speed/executive function, as well as positive correlations between ACG Cr and Glu with visual memory (Table 4).

Table 4.

Partial Correlations between Neurochemistry and Clinical Factor Composite Scores in Symptomatic Former NFL Players

Psychomotor Speed/Exec. Func. Verbal Memory Visual Memory Behavior/Mood
ACG PCG PWM ACG PCG PWM ACG PCG PWM ACG PCG PWM
Cr .09(.31) −.06(.74) .02(0.79) −.15(.28) −.23(.07) −.07(.57) .27(.04) .07(.59) −.14(.50) .19(.11) .07(.57) −.10(.38)
Glx .24(.02) .05(.63) −.09(0.54) −.002(.96) −.06(.48) .13(.37) .25(.05) −.06(.72) .01(.96) .22(.05) .11(.30) −.06(.71)
Glu .18(.14) −.0002(.86) −.04(0.72) −.01(.96) −.12(.25) .03(.94) .26(.02) −.03(.92) −.10(.45) .32(.004) .04(.64) −.001(.98)
Cho .10(.38) .001(.85) .18(0.14) .07(.58) .01(.98) .19(.20) .22(.08) .11(.38) .04(.82) .23(.05) −.002(.98) −.12(.34)
GSH .16(.26) −.14(.32) −.15(0.37) −.07(.68) .02(.82) .09(.34) .11(.44) −.02(.82) −.06(.62) .29(.02) .15(.28) −.13(.19)
mI .08(.42) −.06(.61) −.02(0.95) −.18(.16) −.25(0.06) −.24(.07) .16(.14) −.05(.66) −.11(.41) .26(.01) .08(.47) −.15(.27)
NAA .14(.20) .07(.52) .13(0.34) .08(.46) −.09(.37) .18(.09) .21(.11) −.01(.96) −.21(.22) .42(<.001) .17(.10) .05(.71)

Partial correlations controlling for age, body mass index, and education. P-value is in parenthesis and bolded are significant at the less than 0.05 alpha level. The clinical factor composites were derived using principal component analysis. Bootstrap analysis was performed on 500 replicates to further control for Type I error and increase statistical power. ACG = anterior cingulate gyrus; PCG = posterior cingulate gyrus; PWM = parietal white matter; Cr = creatine; Glx = glutamate/glutamine; Glu = glutamate; Cho = choline; GSH = Glutathione; mI = myo-Inositol; and NAA = N-acetyl aspartate.

Fig 3.

Fig 3.

Association between Anterior Cingulate Gyrus Neurochemistry and Behavioral/Mood Symptoms. The scatter plots are of the residuals between the anterior cingulate gyrus (ACG) glutamate (glu), glutathione (GSH), and myo-inositol (mI) concentrations and the behavioral/mood principal component composite score after controlling for age, body mass index, and education. Higher concentrations of Glu (A), GSH (B), and mI (C) were associated with higher (i.e., worse) behavioral/mood scores. The x-axis is the neurochemical concentrations and the y-axis is the behavioral/mood composite scores. Shaded region represents 95% CI.

DISCUSSION

In this sample of symptomatic former NFL players, there was a direct relationship between greater exposure to RHI (as measured by the CHII) and lower Cr in the PWM. However, only NAA concentrations in the PWM were found to be different (i.e., lower) among the former NFL players compared to the “controls”. In the former NFL players, higher ACG concentrations of neurochemicals that can reflect immunoexcitotoxicity (i.e., Glu) and neuroinflammation (i.e., GSH, mI) corresponded with greater behavioral/mood symptoms. ACG NAA correlated with behavioral/mood symptoms and other ACG neurochemicals were associated with psychomotor speed/executive function and visual memory; however, the directionality of these effects was opposite of expected. This study provides evidence for reduced cellular energy metabolism as a long-term consequence of RHI. The findings additionally provide initial support for neuroinflammation, particularly in the ACG, as a possible contributor to behavioral/mood disturbances in symptomatic former NFL players.

Our previous research has shown the CHII to be associated with later-life cognitive and neuropsychiatric function in former amateur American football players (Montenigro et al. 2017), as well as with higher concentrations of plasma (Alosco, Tripodis, et al. 2017) and cerebrospinal fluid (CSF) (Alosco, Tripodis, et al. 2018) total tau, and greater burden of white matter hyperintensities (Alosco, Koerte, et al. 2018) in former NFL players from the DETECT study. The current study extends these findings by showing an inverse relationship between the CHII and PWM Cr in former NFL players from DETECT. Cr regulates cellular energy metabolism and production of ATP (Brewer & Wallimann 2000), thereby exposure to RHI may be associated with later-life reduced cellular energy metabolism. Although Cr has traditionally been thought to be constant, this may not be the case in setting of head trauma (Gasparovic et al. 2009). The association between the CHII and Cr was only observed in the PWM. The parietal lobe may be more susceptible to later life consequences of RHI, given the specific effects for this region in this sample and other studies of former NFL players that examined molecular and functional imaging (Coughlin et al. 2015; Ford, Giovanello, & Guskiewicz 2013).

The CHII was not associated with other neurochemicals and there were no differences in Cr between the former NFL players and “controls”. Instead, only lower PWM NAA was observed in the former NFL players. NAA is of particular interest because it is produced in neurons and transported down axons, and decreased NAA reflects reduced neuronal, axonal, and dendritic viability. It is thus a marker of neurodegeneration (Lin et al. 2012). Previous work shows acute NAA reductions in active contact sport athletes (Chamard et al. 2012) and decreased NAA in three former boxers with parkinsonism (Davie et al. 1995), when compared to controls and idiopathic Parkinson’s disease. Although, other research did not report differences in NAA between former professional soccer players and non-contact sport athletes (Koerte et al. 2015). Results from the group comparison(s) in the present study have methodological caveats that limit the ability to draw meaningful conclusions. The control group was small and not well matched to the former NFL players on factors such as race. Study eligibility criteria required the former NFL players to be symptomatic at the time of recruitment, whereas the comparison group must have been asymptomatic. The neurochemical alterations may therefore be a result of exposure to RHI or related to differences in symptom status.

Other notable findings included the associations between higher ACG Glu, GSH, and mI with greater behavioral/mood symptoms in the former NFL players. A previous MRS study in former professional soccer players showed that higher mI and GHS concentrations correlated with greater number of lifetime estimate of headings; higher GSH was also associated with worse executive function (Koerte et al. 2015). Glu is the most abundant excitatory neurotransmitter in the brain and excessive Glu can lead to excitotoxicity (Ramadan, Lin, & Stanwell 2013). GSH is an antioxidant that is activated as a compensatory response to oxidative stress and neuroinflammation (Duffy et al. 2014). Increased mI is considered to be a marker of astrocytosis and microglial activation (Fisher, Novak, & Agranoff 2002). Based on retrospective informant reports, the clinical presentation of CTE includes a constellation of cognitive, behavioral, and mood disturbances (Alosco, Mez, et al. 2018; Mez et al. 2017; Montenigro et al. 2014; Stern et al. 2013). Behavioral/mood disturbances tend to begin at a young age compared to a later onset of cognitive impairments (Alosco, Mez, et al. 2018; Mez et al. 2017; Stern et al. 2013). While later onset cognitive impairment is typical with neurodegeneration, it is not clear if neuropsychiatric disturbances in CTE are due to idiopathic mental illness, CTE p-tau deposition, or other types of pathologies. There is accumulating evidence for neuroinflammation as a possible mechanism that exposure to RHI may lead to CTE (Alosco, Tripodis, et al. 2018; Bari et al. 2018; Cherry et al. 2017; Cherry et al. 2016; Coughlin et al. 2017). Here, associations between the CHII with ACG Glu, GSH, or mI did not emerge. A recent study from DETECT similarly did not find a direct effect between the CHII and CSF sTREM2 (a marker of microglial activation) (Alosco, Tripodis, et al. 2018). Brain alterations can begin in youth football (Bahrami et al. 2016), but the course of brain changes and their corresponding relation with clinical function is unknown. Glu may be elevated during active RHI (Bari et al. 2018) and decrease with neurodegenerative disease due to loss of neuronal density (Fayed et al. 2011; Lin, Shic, Enriquez, & Ross 2003). Longitudinal research is needed to clarify the relation between RHI and neuroinflammation and the corresponding clinical consequences.

The current study revealed some unexpected effects. For example, the positive directionality of effects for several of the ACG neurochemicals and clinical function was opposite of expected (i.e., higher NAA and worse behavior/mood, higher Glx and better psychomotor speed/executive function, as well as higher Cr and Glu and better visual memory). In general, the utility of MRS neurochemicals in the setting of RHI exposure is not well understood. Although we implemented methods to attenuate risk for Type I error, this remains a limitation due to the number of analyses performed. There was substantial variability in neurochemical levels across participants, potentially reflecting the heterogeneity of this sample in terms of pathological burden. Metabolite concentrations and their effects on clinical function may depend on disease stage and this is unknown because CTE cannot be diagnosed during life. Notably, the former NFL players only had worse scores on the behavioral/mood composite and not on the cognitive composites compared to the controls.

Additional methodological limitations include the modest sample size, limited RHI variability, and cross-sectional study design. The ACG was the only brain region where MRS neurochemical measures were associated with clinical function. The ACG is a hub of neural networks that modulate cognition, behavior, and mood (Lichenstein, Verstynen, & Forbes 2016). However, brain regions not examined in this study may be more sensitive to RHI and clinical function. The prefrontal cortex is an initial CTE target (McKee et al. 2016; McKee et al. 2013; Mez et al. 2017). Former university level ice hockey and football players have exhibited elevated prefrontal Cho (Tremblay et al. 2013) and prefrontal neurochemical changes in active high school football players have been observed (Bari et al. 2018). Finally, the neurochemicals examined in this study can be impacted by various etiologies. It is plausible that a competing neurodegenerative disease is present, which could explain the conflicting findings of lower NAA in the former NFL players but no association between NAA and RHI.

In summary, this study found evidence for a direct relationship between greater RHI exposure and reduced later-life cellular energy metabolism in symptomatic former NFL players. Findings additionally suggest that the ACG and associated neuroinflammatory processes may have clinical implications among individuals exposed to RHI, particularly in terms of the development of behavioral and mood disturbances. Yet, the mechanisms that underpin RHI, CTE, and clinical dysfunction are unknown and likely complex, multifaceted, and evolve over time. Although CTE is defined by its unique p-tau deposition, many cases have other non-tau pathologies (e.g., neuroinflammation) that may have commenced during active RHI exposure and lead to neurodegeneration through interaction with other risk factors (e.g., genetics). Further research is needed to better understand the utility of MRS for the study of disease mechanisms in the setting of RHI and CTE.

Funding:

This study was supported by grants from the NIH (P30 AG13846; R01 NS 078337; R56 9500304025; U01 NS093334). This publication was also supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through BU-CTSI Grant Number 1UL1TR001430. Michael L. Alosco, and research reported in this publication, was supported by the National Institutes of Health under grant number F32NS096803 and K23NS102399 and from a Pilot Grant from the Boston University Alzheimer’s Disease and CTE Center (P30AG13846). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

He receives royalties from book publications, and compensation from expert legal opinion. Ann C. McKee has received funding from the NFL, World Wrestling Entertaintment (WWE), and is a member of the Mackey-White Committee of the NFL Players Association. Robert A. Stern has received research funding from Avid Radiopharmaceuticals, Inc. (Philadelphia, PA, USA). He is a member of the Mackey-White Committee of the NFL Players Association. He is a paid consultant to Biogen (Cambridge, MA) and Eli Lilly (Indianapolis, IN), and is a paid member of the Medical Science Committee for the NCAA Student-Athlete Concussion Injury Litigation. He receives royalties for published neuropsychological tests from Psychological Assessment Resources, Inc. (Lutz, FL). Alexander Lin was responsible for study concept and design, revising the manuscript, and analysis and interpretation of data. He is a paid consultant for Agios pharmaceuticals and Moncton MRI. He is also co-founder of Brainspec Inc. The remaining authors have no conflicts of interest to report. All authors have given final approval of the version to be published and agree to be accountable for the work.

Footnotes

Conflict of Interest: Robert C. Cantu is a paid consultant to the National Football League (NFL) Head Neck and Spine Committee and National Operating Committee on Standards for Athletic Equipment (NOCSAE), and is a paid member of the Medical Science Committee for the National Collegiate Athletic Association (NCAA) Student-Athlete Concussion Injury Litigation.

Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Contributor Information

Michael L. Alosco, Boston University Alzheimer’s Disease and CTE Center, Department of Neurology, Boston University School of Medicine, Boston, MA, USA;.

Yorghos Tripodis, Department of Biostatistics, Boston University School of Public Health, Boston, MA, USA;.

Benjamin Rowland, Center for Clinical Spectroscopy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA;.

Alicia S. Chua, Department of Biostatistics, Boston University School of Public Health, Boston, MA, USA;.

Huijun Liao, Center for Clinical Spectroscopy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA;.

Brett Martin, Boston University Alzheimer’s Disease and CTE Center, Boston University School of Medicine; Data Coordinating Center, Boston University School of Public Health, Boston, MA, USA;.

Johnny Jarnagin, Boston University Alzheimer’s Disease and CTE Center, Department of Neurology, Boston University School of Medicine, Boston, MA, USA;.

Christine E. Chaisson, Boston University Alzheimer’s Disease and CTE Center, Department of Neurology, Boston University School of Medicine; Department of Biostatistics, Data Coordinating Center, Boston University School of Public Health, Boston, MA, USA;.

Ofer Pasternak, Departments of Psychiatry and Radiology, Psychiatry Neuroimaging Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA;.

Sarina Karmacharya, Department of Psychiatry, Psychiatry Neuroimaging Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA;.

Inga K. Koerte, Department of Psychiatry, Psychiatry Neuroimaging Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; Department of Child and Adolescent Psychiatry, Psychosomatic, and Psychotherapy, Ludwig-Maximilian-University, Munich Germany;.

Robert C. Cantu, Boston University Alzheimer’s Disease and CTE Center, Departments of Neurology and Neurosurgery, Boston University School of Medicine, Boston, MA, USA; Concussion Legacy Foundation;.

Neil W. Kowall, Boston University Alzheimer’s Disease and CTE Center, Departments of Neurology, and Pathology and Laboratory Medicine, Boston University School of Medicine; Neurology Service, VA Boston Healthcare System, U.S. Department of Veteran Affairs, Boston, MA, USA;.

Ann C. McKee, Boston University Alzheimer’s Disease and CTE Center, Departments of Neurology, and Pathology and Laboratory Medicine, Boston University School of Medicine, Boston, MA, USA; VA Boston Healthcare System, U.S. Department of Veteran Affairs; Department of Veterans Affairs Medical Center, Bedford;.

Martha E. Shenton, Departments of Psychiatry and Radiology, Psychiatry Neuroimaging Laboratory, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA; VA Boston Healthcare System, U.S. Department of Veteran Affairs;.

Richard Greenwald, Simbex, Lebanon, NH; Thayer School of Engineering, Dartmouth College, Hanover, NH;.

Michael McClean, Department of Environmental Health, Boston University School of Public Health, Boston, MA, USA;.

Robert A. Stern, Boston University Alzheimer’s Disease and CTE Center, Departments of Neurology, Neurosurgery, and Anatomy & Neurobiology, Boston University School of Medicine, Boston, MA, USA;.

Alexander Lin, Center for Clinical Spectroscopy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA;.

REFERENCES

  1. Alosco ML, Jarnagin J, Tripodis Y, Platt M, Martin B, Chaisson CE, et al. (2017). Olfactory Function and Associated Clinical Correlates in Former National Football League Players. J Neurotrauma, 34(4), 772–780. doi: 10.1089/neu.2016.4536 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alosco ML, Koerte IK, Tripodis Y, Mariani M, Chua AS, Jarnagin J, et al. (2018). White matter signal abnormalities in former National Football League players. Alzheimers Dement (Amst), 10, 56–65. doi: 10.1016/j.dadm.2017.10.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Alosco ML, Mez J, Tripodis Y, Kiernan PT, Abdolmohammadi B, Murphy L, et al. (2018). Age of First Exposure to Tackle Football and Chronic Traumatic Encephalopathy. Ann Neurol. doi: 10.1002/ana.25245 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alosco ML, Tripodis Y, Fritts NG, Heslegrave A, Baugh CM, Conneely S, et al. (2018). Cerebrospinal fluid tau, Abeta, and sTREM2 in Former National Football League Players: Modeling the relationship between repetitive head impacts, microglial activation, and neurodegeneration. Alzheimers Dement, 14(9), 1159–1170. doi: 10.1016/j.jalz.2018.05.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Alosco ML, Tripodis Y, Jarnagin J, Baugh CM, Martin B, Chaisson CE, et al. (2017). Repetitive head impact exposure and later-life plasma total tau in former National Football League players. Alzheimers Dement (Amst), 7, 33–40. doi: 10.1016/j.dadm.2016.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bahrami N, Sharma D, Rosenthal S, Davenport EM, Urban JE, Wagner B, et al. (2016). Subconcussive Head Impact Exposure and White Matter Tract Changes over a Single Season of Youth Football. Radiology, 281(3), 919–926. doi: 10.1148/radiol.2016160564 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bari S, Svaldi DO, Jang I, Shenk TE, Poole VN, Lee T, et al. (2018). Dependence on subconcussive impacts of brain metabolism in collision sport athletes: an MR spectroscopic study. Brain Imaging Behav. doi: 10.1007/s11682-018-9861-9 [DOI] [PubMed] [Google Scholar]
  8. Brewer GJ, & Wallimann TW (2000). Protective effect of the energy precursor creatine against toxicity of glutamate and beta-amyloid in rat hippocampal neurons. J Neurochem, 74(5), 1968–1978. [DOI] [PubMed] [Google Scholar]
  9. Chamard E, Theoret H, Skopelja EN, Forwell LA, Johnson AM, & Echlin PS (2012). A prospective study of physician-observed concussion during a varsity university hockey season: metabolic changes in ice hockey players. Part 4 of 4. Neurosurg Focus, 33(6), E4: 1–7. doi: 10.317/2012.10.FOCUS12305 [DOI] [PubMed] [Google Scholar]
  10. Cherry JD, Stein TD, Tripodis Y, Alvarez VE, Huber BR, Au R, et al. (2017). CCL11 is increased in the CNS in chronic traumatic encephalopathy but not in Alzheimer’s disease. PLoS One, 12(9), e0185541. doi: 10.1371/journal.pone.0185541 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cherry JD, Tripodis Y, Alvarez VE, Huber B, Kiernan PT, Daneshvar DH, et al. (2016). Microglial neuroinflammation contributes to tau accumulation in chronic traumatic encephalopathy. Acta Neuropathol Commun, 4(1), 112. doi: 10.1186/s40478-016-0382-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Coughlin JM, Wang Y, Minn I, Bienko N, Ambinder EB, Xu X, et al. (2017). Imaging of Glial Cell Activation and White Matter Integrity in Brains of Active and Recently Retired National Football League Players. JAMA Neurol, 74(1), 67–74. doi: 10.1001/jamaneurol.2016.3764 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Coughlin JM, Wang Y, Munro CA, Ma S, Yue C, Chen S, et al. (2015). Neuroinflammation and brain atrophy in former NFL players: An in vivo multimodal imaging pilot study. Neurobiol Dis, 74, 58–65. doi: 10.1016/j.nbd.2014.10.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Davie CA, Pirtosek Z, Barker GJ, Kingsley DP, Miller PH, & Lees AJ (1995). Magnetic resonance spectroscopic study of parkinsonism related to boxing. J Neurol Neurosurg Psychiatry, 58(6), 688–691. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Duffy SL, Lagopoulos J, Hickie IB, Diamond K, Graeber MB, Lewis SJ, et al. (2014). Glutathione relates to neuropsychological functioning in mild cognitive impairment. Alzheimers Dement, 10(1), 67–75. doi: 10.1016/j.jalz.2013.01.005 [DOI] [PubMed] [Google Scholar]
  16. Ernst T, Ross BD (1993). Absolute quantitation of water and metabolites in the human brain. I. Compartments and water. Journal of Magnetic Resonance, Series B, 102, 1–8. [Google Scholar]
  17. Fayed N, Modrego PJ, Rojas-Salinas G, & Aguilar K (2011). Brain glutamate levels are decreased in Alzheimer’s disease: a magnetic resonance spectroscopy study. Am J Alzheimers Dis Other Demen, 26(6), 450–456. doi: 10.1177/1533317511421780 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fisher SK, Novak JE, & Agranoff BW (2002). Inositol and higher inositol phosphates in neural tissues: homeostasis, metabolism and functional significance. J Neurochem, 82(4), 736–754. [DOI] [PubMed] [Google Scholar]
  19. Ford JH, Giovanello KS, & Guskiewicz KM (2013). Episodic memory in former professional football players with a history of concussion: an event-related functional neuroimaging study. J Neurotrauma, 30(20), 1683–1701. doi: 10.1089/neu.2012.2535 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gardner AJ, Iverson GL, Wojtowicz M, Levi CR, Kay-Lambkin F, Schofield PW, et al. (2017). MR Spectroscopy Findings in Retired Professional Rugby League Players. Int J Sports Med, 38(3), 241–252. doi: 10.1055/s-0042-120843 [DOI] [PubMed] [Google Scholar]
  21. Gasparovic C, Yeo R, Mannell M, Ling J, Elgie R, Phillips J, et al. (2009). Neurometabolite concentrations in gray and white matter in mild traumatic brain injury: an 1H-magnetic resonance spectroscopy study. J Neurotrauma, 26(10), 1635–1643. doi: 10.1089/neu.2009-0896 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Graff-Radford J, & Kantarci K (2013). Magnetic resonance spectroscopy in Alzheimer’s disease. Neuropsychiatr Dis Treat, 9, 687–696. doi: 10.2147/NDT.S35440 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kantarci K, Weigand SD, Petersen RC, Boeve BF, Knopman DS, Gunter J, et al. (2007). Longitudinal 1H MRS changes in mild cognitive impairment and Alzheimer’s disease. Neurobiol Aging, 28(9), 1330–1339. doi: 10.1016/j.neurobiolaging.2006.06.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Koerte IK, Lin AP, Muehlmann M, Merugumala S, Liao H, Starr T, et al. (2015). Altered Neurochemistry in Former Professional Soccer Players without a History of Concussion. J Neurotrauma, 32(17), 1287–1293. doi: 10.1089/neu.2014.3715 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lichenstein SD, Verstynen T, & Forbes EE (2016). Adolescent brain development and depression: A case for the importance of connectivity of the anterior cingulate cortex. Neurosci Biobehav Rev, 70, 271–287. doi: 10.1016/j.neubiorev.2016.07.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lin AP, Liao HJ, Merugumala SK, Prabhu SP, Meehan WP 3rd, & Ross BD (2012). Metabolic imaging of mild traumatic brain injury. Brain Imaging Behav, 6(2), 208–223. doi: 10.1007/s11682-012-9181-4 [DOI] [PubMed] [Google Scholar]
  27. Lin AP, Ramadan S, Stern RA, Box HC, Nowinski CJ, Ross BD, et al. (2015). Changes in the neurochemistry of athletes with repetitive brain trauma: preliminary results using localized correlated spectroscopy. Alzheimers Res Ther, 7(1), 13. doi: 10.1186/s13195-015-0094-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lin AP, Shic F, Enriquez C, & Ross BD (2003). Reduced glutamate neurotransmission in patients with Alzheimer’s disease -- an in vivo (13)C magnetic resonance spectroscopy study. MAGMA, 16(1), 29–42. doi: 10.1007/s10334-003-0004-x [DOI] [PubMed] [Google Scholar]
  29. McKee AC, Cairns NJ, Dickson DW, Folkerth RD, Keene CD, Litvan I, et al. (2016). The first NINDS/NIBIB consensus meeting to define neuropathological criteria for the diagnosis of chronic traumatic encephalopathy. Acta Neuropathol, 131(1), 75–86. doi: 10.1007/s00401-015-1515-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McKee AC, Stern RA, Nowinski CJ, Stein TD, Alvarez VE, Daneshvar DH, et al. (2013). The spectrum of disease in chronic traumatic encephalopathy. Brain, 136(Pt 1), 43–64. doi: 10.1093/brain/aws307 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mez J, Daneshvar DH, Kiernan PT, Abdolmohammadi B, Alvarez VE, Huber BR, et al. (2017). Clinicopathological Evaluation of Chronic Traumatic Encephalopathy in Players of American Football. JAMA, 318(4), 360–370. doi: 10.1001/jama.2017.8334 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Montenigro PH, Alosco ML, Martin BM, Daneshvar DH, Mez J, Chaisson CE, et al. (2017). Cumulative Head Impact Exposure Predicts Later-Life Depression, Apathy, Executive Dysfunction, and Cognitive Impairment in Former High School and College Football Players. J Neurotrauma, 34(2), 328–340. doi: 10.1089/neu.2016.4413 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Montenigro PH, Baugh CM, Daneshvar DH, Mez J, Budson AE, Au R, et al. (2014). Clinical subtypes of chronic traumatic encephalopathy: literature review and proposed research diagnostic criteria for traumatic encephalopathy syndrome. Alzheimers Res Ther, 6(5), 68. doi: 10.1186/s13195-014-0068-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Ramadan S, Lin A, & Stanwell P (2013). Glutamate and glutamine: a review of in vivo MRS in the human brain. NMR Biomed, 26(12), 1630–1646. doi: 10.1002/nbm.3045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Rowland BC, Liao H, Adan F, Mariano L, Irvine J, & Lin AP (2017). Correcting for Frequency Drift in Clinical Brain MR Spectroscopy. J Neuroimaging, 27(1), 23–28. doi: 10.1111/jon.12388 [DOI] [PubMed] [Google Scholar]
  36. Sainani K (2010). The importance of accounting for correlated observations. PM R, 2(9), 858–861. doi: 10.1016/j.pmrj.2010.07.482 [DOI] [PubMed] [Google Scholar]
  37. Stamm JM, Bourlas AP, Baugh CM, Fritts NG, Daneshvar DH, Martin BM, et al. (2015). Age of first exposure to football and later-life cognitive impairment in former NFL players. Neurology, 84(11), 1114–1120. doi: 10.1212/WNL.0000000000001358 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Stamm JM, Koerte IK, Muehlmann M, Pasternak O, Bourlas AP, Baugh CM, et al. (2015). Age at First Exposure to Football Is Associated with Altered Corpus Callosum White Matter Microstructure in Former Professional Football Players. J Neurotrauma, 32(22), 1768–1776. doi: 10.1089/neu.2014.3822 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stern RA, Daneshvar DH, Baugh CM, Seichepine DR, Montenigro PH, Riley DO, et al. (2013). Clinical presentation of chronic traumatic encephalopathy. Neurology, 81(13), 1122–1129. doi: 10.1212/WNL.0b013e3182a55f7f [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Tremblay S, De Beaumont L, Henry LC, Boulanger Y, Evans AC, Bourgouin P, et al. (2013). Sports concussions and aging: a neuroimaging investigation. Cereb Cortex, 23(5), 1159–1166. doi: 10.1093/cercor/bhs102 [DOI] [PubMed] [Google Scholar]
  41. Watanabe T, Shiino A, & Akiguchi I (2012). Hippocampal metabolites and memory performances in patients with amnestic mild cognitive impairment and Alzheimer’s disease. Neurobiol Learn Mem, 97(3), 289–293. doi: 10.1016/j.nlm.2012.01.006 [DOI] [PubMed] [Google Scholar]

RESOURCES