Abstract
Introduction
We determined whether the effect of apolipoprotein E (APOE)-ε4 genotype on Alzheimer's disease (AD) markers differs in men and women across AD stages.
Methods
Among normal control (NC) participants (N = 702) and participants with mild cognitive impairment (N = 576) and AD (N = 305), we examined the associations of sex and APOE-ε4 carrier status with cortical amyloid-β (Aβ) burden, hippocampal volume ratio (HpVR; hippocampal volume/intracranial volume × 103), brain glucose metabolism, and verbal memory.
Results
In NC, APOE-ε4 related to greater Aβ burden and poorer verbal memory across sex but to smaller HpVR and hypometabolism in men only. In mild cognitive impairment, APOE-ε4 related to smaller HpVR, hypometabolism, greater Aβ burden, and poorer verbal memory across sex. In AD, APOE-ε4 related to greater Aβ burden in men only and smaller HpVR across sex and showed no association with hypometabolism or verbal memory.
Discussion
Sex differences in the association between APOE-ε4 and AD markers vary by disease stage.
Keywords: APOE, Sex differences, Amyloid-β plaque deposition, Hippocampal volume, Brain glucose metabolism, Verbal memory
1. Background
The ε4 allele of the apolipoprotein E gene (APOE-ε4) is the most common genetic risk factor for Alzheimer's disease (AD) [1], [2]. APOE-ε4 is associated with AD-related biological and clinical markers including cortical amyloid-β (Aβ) plaque burden, hippocampal atrophy [3], [4], [5], and accelerated cognitive decline in healthy aging [6], [7], [8]. However, there have been conflicting/mixed findings with some reporting that APOE-ε4 is not associated with hippocampal volume [9], [10] or cognitive decline in healthy aging [9], [10] or with risk of converting from mild cognitive impairment (MCI) to AD [11], [12]. These inconsistencies may be due to the potentially critical modulating role of sex in the association between APOE-ε4 and AD.
Early cross-sectional studies indicated that APOE-ε4 confers a greater risk for AD in women than in men [13], [14], [15]. These early findings are supported by a longitudinal study [16] and a meta-analysis that reported an increased risk of incident AD in female APOE-ε4 heterozygotes versus male heterozygotes among adults aged 65 to 75 years [17]. Among older normal controls, the adverse effect of APOE-ε4 on AD biomarkers including cerebrospinal fluid total tau levels [18], brain metabolism [19], cortical thinning [19], and functional brain connectivity in the default mode network [18] was stronger in women versus men, although not consistently [19], [20]. In MCI, the effect of APOE-ε4 on brain atrophy [21], total tau levels [16], and the tau/Aβ ratio [16] was stronger in women versus men. To our knowledge, only one study examined sex differences in the effects of APOE-ε4 in AD patients and reported a stronger association between APOE-ε4 and Aβ burden in parietal, cingulate, and frontal regions in men versus women [22], suggesting a possible reversal in the moderating role of sex on APOE-ε4 in the MCI-to-AD transition that warrants further exploration.
Surprisingly, few studies have examined the interactive effects of sex and APOE-ε4 on cognitive performance [21], [23], [24], particularly verbal memory, even though verbal memory is the cognitive domain that shows the earliest and most severe deficits in AD [25]. Among community-dwelling older adults, the association between APOE-ε4 and accelerated cognitive decline, as measured by a global cognitive measure, was stronger in women versus men [23], [24]. In a MCI sample, female heterozygous or homozygous APOE-ε4 carriers showed worse performance on a delayed (5 minute) word recall task from the Alzheimer's Disease Assessment Scale Cognitive Subscale compared with female noncarriers, whereas the APOE-ε4 effect was only evident in homozygous men [21]. In addition, few studies have examined the sex by APOE interaction across the AD continuum [22] despite evidence of a temporal ordering of AD markers, whereby Aβ deposition occurs first, followed by neurodegenerative biomarkers and cognitive impairment [26], [27]. We systematically examined the separate and interactive effects of sex and APOE on multiple AD-related markers including Aβ deposition, hippocampal volume, brain glucose metabolism, and verbal memory performance in the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Effects were examined in each disease stage (NC, MCI, AD), in which MCI versus NC was defined using the Jak/Bondi diagnostic method, an actuarial, neuropsychological diagnostic approach that has produced more discernible cognitive phenotypes, more stable diagnoses, stronger associations with AD biomarkers, and better predication of progression to dementia than conventional diagnostic criteria [28], [29]. Consistent with the broader literature, we predicted that the adverse effects of APOE-ε4 would be stronger in women versus men, and in-line with the temporal sequence of AD-related markers [26], [27], this sex difference in the APOE-ε4 effect will manifest at the NC stage for earlier AD events (Aβ deposition) and in MCI for later events (hippocampal atrophy, brain hypometabolism, and memory deficits).
2. Methods
2.1. Participants and data source
Data were extracted from the ADNI database (adni.loni.usc.edu). ADNI is a longitudinal, multisite cohort study that began in 2003 as a public-private partnership. Information about ADNI can be found at www.adni-info.org. ADNI study visits involve neuroimaging, neuropsychological, and clinical assessments. We included participants who had APOE genotype and baseline data on one of the AD-related markers examined herein. We limited our sample to ADNI's largest race/ethnic group, Caucasians, to minimize potential population stratification bias that could complicate interpretation of genetic data. Analyses were repeated while excluding APOE-ε2 allele carriers because the protective effect of APOE-ε2 could mask the adverse effect of APOE-ε4.
2.2. Verbal memory assessment
Our verbal memory measure was the Rey Auditory Verbal Learning Test (AVLT) [30]. The AVLT is a multitrial list learning and memory test with immediate and delayed recall and recognition outcomes. Immediate recall scores (range: 0–75) were the primary outcome because they were not used in diagnostic criteria, and learning deficits may better discriminate preclinical AD from normal controls than retention deficits [31], [32].
2.3. Biomarkers
Biomarkers included neuroimaging measures of hippocampal volume, brain glucose metabolism, and cortical Aβ deposition. Structural MRI scans were collected on a 1.5T scanner according to a standardized protocol [33]. Hippocampal volume data were analyzed using FreeSurfer version 4.3 (https://surfer.nmr.mgh.harvard.edu) at the University of California–San Francisco (http://adni.loni.ucla.edu/wp-content/uploads/2010/12/UCSF-FreeSurfer-Overview-and-QC_-Template_Format.pdf) [34]. To control for sex differences in head size, we calculated a hippocampal volume ratio (HpVR) using the formula, hippocampal/intracranial volume × 103.
Brain glucose metabolism was measured by [18F]fludeoxyglucose (FDG) positron emission tomography (PET). Images were preprocessed following a standard procedure described in http://adni.loni.usc.edu/methods/pet-analysis/pre-processing/. ADNI investigators at the University of California, Berkeley, established a “MetaROI” of brain regions that commonly demonstrate metabolic changes in MCI/AD which correlate with cognitive performance in a meta-analysis [11], [35]. The “MetaROI” was comprised of bilateral posterior cingulate gyrus, bilateral angular gyri, and middle/inferior temporal gyrus. Standardized uptake value ratios (SUVRs) were calculated by averaging FDG uptake across the MetaROI and dividing by a reference region of pons and cerebellum [11], [35]. FDG PET image analysis is described in http://www.adni-info.org/Scientists/ADNIStudyProcedures.aspx.
Cortical Aβ burden was measured by [18F]florbetapir positron emission tomography (AV45 PET) as described in http://www.adni-info.org. Mean AV45 uptake was measured within frontal, anterior/posterior cingulate, lateral parietal, and lateral temporal regions. SUVRs were calculated by averaging across regions and dividing by whole cerebellum.
2.4. Diagnostic classification
Diagnosis of NC versus MCI was based on the Jak/Bondi diagnostic method [28]. This method included six neuropsychological tests representing three cognitive domains: (1) Trail-Making Tests A and B (psychomotor speed/executive function); (2) Category Fluency and Boston Naming Test (language); and (3) AVLT Delayed Recall and Recognition Tests (episodic memory). An impaired score was defined as >1 SD below the age-corrected normative mean. MCI diagnosis required one of three criteria: (1) impaired score on two tests within a cognitive domain; (2) one impaired score in each of the three cognitive domains; and/or (3) a score of 9 on the Functional Assessment Questionnaire indicating dependence in at least three daily activities. If no criterion was met, a NC diagnosis was provided. Diagnostic criteria for AD were based on the standard National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association criteria [36].
2.5. Statistical analysis
The APOE genotype was categorized into APOE-ε4 carriers and noncarriers because of the low prevalence of APOE-ε4 homozygotes when stratifying by sex and diagnostic group (e.g., 8 women and 22 men in NC group). Analyses were conducted within diagnostic group. Differences in sample characteristics and AD-related markers between sex and APOE-ε4 status were examined using independent t-tests for continuous variables and Chi-square tests for categorical variables. We used multivariable linear regression analyses modeling the separate and interactive associations of sex and APOE-ε4 status with each AD-associated marker (HpVR, FDG SUVR, AV45 SUVR, and AVLT scores) while adjusting for age and education. The interaction was removed from the model if P ≥ .10. Analyses were performed using SPSS 24 (SPSS Inc., Chicago, IL). Significance was defined as P ≤ .05 (two-sided).
3. Results
Our sample included 1583 participants with AVLT scores, 1370 with HpVR, 998 with FDG SUVR, and 820 with AV45 SUVR. Table 1 displays sample sizes by sex and APOE-ε4 status for each diagnostic group and AD-related marker. In the largest sample (participants with AVLT scores), there were no sex differences in the distribution of APOE-ε4 carriers in any diagnostic group. In all diagnostic groups, women were younger and had less years of education than men (P's < .05; Table 2). Mean Mini-Mental State Examination score was higher in NC women versus NC men (P = .002), although the difference was only half a point. APOE-ε4 carriers were younger than noncarriers in all diagnostic groups (P < .01). APOE-ε4 carriers had a lower mean Mini-Mental State Examination and a higher mean Clinical Dementia Rating–Sum of Boxes socres than noncarriers in MCI (P < .001). Differences between sex and APOE-ε4 status groups on AD-related markers are examined in the linear regressions that are described in the following.
Table 1.
Diagnostic group/outcome | Total sample |
APOE-ε4+ |
APOE-ε4− |
||
---|---|---|---|---|---|
Women | Men | Women | Men | ||
NC | |||||
AVLT Immediate Recall | 702 | 107 | 126 | 228 | 241 |
HpVR | 630 | 101 | 109 | 205 | 215 |
FDG SUVR | 562 | 88 | 101 | 180 | 193 |
AV45 SUVR | 430 | 78 | 71 | 143 | 138 |
MCI | |||||
AVLT Immediate Recall | 576 | 127 | 183 | 101 | 165 |
HpVR | 490 | 112 | 152 | 84 | 142 |
FDG SUVR | 414 | 87 | 135 | 73 | 119 |
AV45 SUVR | 260 | 58 | 78 | 52 | 72 |
AD | |||||
AVLT Immediate Recall | 305 | 90 | 120 | 45 | 58 |
HpVR | 250 | 73 | 97 | 36 | 44 |
FDG SUVR | 220 | 63 | 83 | 25 | 49 |
AV45 SUVR | 130 | 39 | 48 | 15 | 28 |
Abbreviations: AD, Alzheimer's disease; APOE-ε4, apolipoprotein ε4 allele; AV45, florbetapir positron emission tomography; AVLT, Rey Auditory Verbal Learning Test; FDG, fluodeoxyglucose; HpVR, hippocampal volume ratio (hippocampal/intracranial volume × 103); MCI, mild cognitive impairment; NC, normal control; SUVR, standardized uptake value ratio.
Table 2.
Parameters | Sex |
Women versus men P value |
APOE-ε4 status |
APOE-ε4+ versus APOE-ε4− P value | ||
---|---|---|---|---|---|---|
Women | Men | APOE-ε4+ | APOE-ε4− | |||
NC | ||||||
Age | 73.0 (6.6) | 74.1 (6.9) | .01 | 72.4 (6.9) | 74.1 (6.7) | .001 |
Education, years | 15.8 (2.6) | 17.0 (2.5) | <.001 | 16.2 (2.5) | 16.5 (2.6) | ns |
MMSE | 28.9 (1.3) | 28.6 (1.5) | .002 | 28.6 (1.6) | 28.8 (1.3) | ns |
AVLT Immediate Recall | 47.7 (8.9) | 44.1 (10.1) | <.001 | 43.4 (10.6) | 44.5 (9.8) | ns |
HpVR | 5.0 (0.6) | 4.7 (0.7) | <.001 | 4.8 (0.7) | 4.9 (0.7) | ns |
FDG SUVR | 1.32 (0.11) | 1.29 (0.12) | <.001 | 1.28 (0.12) | 1.31 (0.11) | .002 |
AV45 SUVR | 1.2 (0.2) | 1.1 (0.2) | .04 | 1.2 (0.2) | 1.1 (0.2) | <.001 |
MCI | ||||||
Age | 72.5 (7.4) | 74.5 (6.8) | .003 | 72.9 (6.7) | 74.6 (7.4) | <.001 |
Education, years | 15.4 (2.8) | 16.2 (2.8) | .003 | 15.8 (2.9) | 15.9 (2.7) | ns |
MMSE | 27.4 (1.9) | 27.4 (1.8) | ns | 27.2 (1.8) | 27.7 (1.8) | <.001 |
CDR-SOB | 1.5 (1.0) | 1.6 (1.0) | ns | 1.7 (1.0) | 1.4 (1.0) | <.001 |
AVLT Immediate Recall | 32.6 (8.7) | 29.3 (7.5) | <.001 | 29.5 (7.7) | 31.9 (8.5) | <.001 |
HpVR | 4.4 (0.8) | 4.2 (0.7) | .005 | 4.2 (0.8) | 4.4 (0.7) | .02 |
FDG SUVR | 1.22 (0.14) | 1.22 (0.13) | ns | 1.20 (0.13) | 1.24 (0.13) | .002 |
AV45 SUVR | 1.2 (0.2) | 1.3 (0.2) | ns | 1.3 (0.2) | 1.1 (0.2) | <.001 |
AD | ||||||
Age | 73.8 (7.9) | 76.0 (7.6) | .03 | 74.1 (7.2) | 76.8 (8.6) | .005 |
Education, years | 14.4 (2.6) | 16.0 (2.8) | <.001 | 15.2 (2.8) | 15.5 (2.8) | ns |
MMSE | 23.3 (2.1) | 23.2 (2.0) | ns | 23.4 (2.0) | 23.1 (2.1) | ns |
CDR-SOB | 4.5 (1.7) | 4.4 (1.6) | ns | 4.3 (1.6) | 4.6 (1.8) | ns |
AVLT Immediate Recall | 23.8 (7.9) | 21.5 (6.8) | .03 | 22.8 (7.0) | 21.9 (8.0) | ns |
HpVR | 3.9 (0.7) | 3.7 (0.6) | .02 | 3.7 (0.5) | 3.9 (0.8) | .05 |
FDG SUVR | 1.06 (0.15) | 1.07 (0.14) | ns | 1.07 (0.14) | 1.07 (0.15) | ns |
AV45 SUVR | 1.5 (0.2) | 1.4 (0.2) | .001 | 1.5 (0.2) | 1.3 (0.3) | <.001 |
Abbreviations: AD, Alzheimer's disease; APOE-ε4, apolipoprotein ε4 allele; AV45, florbetapir positron emission tomography; AVLT, Rey Auditory Verbal Learning Test; CDR-SOB, Clinical Dementia Rating–Sum of Boxes; FDG, fluodeoxyglucose; HpVR, hippocampal volume ratio (hippocampal/intracranial volume × 103); MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; NC, normal control; SUVR, standardized uptake value ratio.
NOTE. Values are expressed as mean (standard deviation). Sample characteristics are based on the largest sample (participants with AVLT Immediate Recall data). The pattern of sex and APOE-ε4 status differences in sample characteristics was similar in all AD marker subsamples.
3.1. Normal control
Inconsistent with hypotheses, there was a significant sex by APOE interaction on HpVR (Table 3) and FDG SUVR in NC (Fig. 1). The APOE-ε4 allele was associated with smaller HpVR and lower FDG SUVR in men (HpVR: B = −0.26, SE = 0.07, P < .001; FDG SUVR: B = −0.06, SE = 0.02, P < .001) but not in women (HpVR: B = 0.01, SE = 0.07, P = .86; FDG SUVR: B = −0.01, SE = 0.01, P = .33). The interaction was not significant for AVLT scores or AV45 SUVR. Rather, there was a main effect of sex and APOE-ε4 for both markers in NC; however, as seen in Fig. 1D, the adverse effect of APOE-ε4 on AVLT scores was only significant among NC men in sex-stratified analyses. Women demonstrated higher AVLT scores and AV45 SUVR than men (P ≤ .001), and APOE-ε4 carriers demonstrated higher AV45 SUVR and lower AVLT scores than noncarriers (P < .05).
Table 3.
Sample/outcome | Multivariable linear regression models |
|||||
---|---|---|---|---|---|---|
Sex (male versus female) |
APOE-ε4 (ε4 carriers versus noncarriers) status |
Sex × APOE-ε4 status |
||||
B, β (SE) | P value | B, β (SE) | P value | B, β (SE) | P value | |
NC | ||||||
HpVR | −0.16, −0.12 (0.06) | .007 | 0.02, 0.01 (0.07) | .81 | −0.28, −0.16 (0.1) | .004 |
FDG SUVR | −0.02, −0.10 (0.01) | .04 | −0.02, −0.06 (0.01) | .29 | −0.04, −0.15 (0.02) | .02 |
AV45 SUVR | −0.04, −0.18 (0.01) | <.001 | −0.03, −0.18 (0.01) | .002 | 0.01, 0.02 (0.03) | .78 |
AVLT Immediate Recall | −7.33, −0.36 (0.69) | <.001 | −1.46, −0.07 (0.71) | .04 | −1.55, −0.06 (1.42) | .27 |
MCI | ||||||
HpVR | 0.01, 0.03 (0.01) | .60 | −0.25, −0.18 (0.01) | <.001 | 0.05, 0.03 (0.13) | .69 |
FDG SUVR | −0.01, −0.09 (0.02) | .22 | −0.08, −0.30 (0.02) | <.001 | 0.04, 0.15 (0.03) | .11 |
AV45 SUVR | 0.00, 0.00 (0.03) | .99 | 0.21, 0.45 (0.03) | <.001 | −0.09, −0.18 (0.05) | .09 |
AVLT Immediate Recall | −3.29, −0.20 (0.67) | <.001 | −2.79, −0.17 (0.65) | <.001 | 0.39, 0.02 (1.33) | .77 |
AD | ||||||
HpVR | −0.20, −0.16 (0.08) | .008 | −0.23, −0.17 (0.08) | .004 | 0.05, 0.04 (0.16) | .76 |
FDG SUVR | −0.01, −0.04 (0.02) | .55 | 0.02, 0.05 (0.02) | .40 | −0.01, −0.05 (0.04) | .74 |
AV45 SUVR | −0.21, −0.49 (0.06) | .001 | 0.03, 0.07 (0.06) | .61 | 0.20, 0.45 (0.07) | .007 |
AVLT Immediate Recall | −2.70, −0.18 (0.89) | .003 | 1.04, 0.07 (0.90) | .25 | −2.29, −0.15 (1.78) | .20 |
Abbreviations: AD, Alzheimer's disease; APOE-ε4, apolipoprotein ε4 allele; AV45 PET, florbetapir positron emission tomography; AVLT, Rey Auditory Verbal Learning Test; B, unstandardized regression coefficient; FDG, fluodeoxyglucose; HpVR, hippocampal volume ratio (hippocampal/intracranial volume × 103); MCI, mild cognitive impairment; NC, normal controls; SUVR, standardized uptake value ratio; β, standardized regression coefficient.
NOTE. All analyses were adjusted for age and education. Bolded text indicates statistical significance at P < .05.
3.2. Mild cognitive impairment
In the MCI sample, there were no significant sex by APOE interactions on any AD-related marker (Fig. 2). There was a main effect of sex on AVLT scores in MCI whereby scores were higher in women than in men (P < .001). There was a main effect of APOE-ε4 for all markers in MCI. APOE-ε4 was associated with smaller HpVR, lower FDG SUVR, higher AV45 SUVR, and poorer AVLT scores (P's < .001).
3.3. Alzheimer's disease
Among AD patients, there was a significant sex by APOE interaction on AV45 SUVR (P = .007; Fig. 3). APOE-ε4 was associated with a higher AV45 SUVR in men (B = 0.23, SE = 0.05, P < .001) but not in women (B = 0.02, SE = 0.05, P = .67). The interaction was not significant for the other markers. As in the other diagnostic groups, there was a main effect of sex on AVLT scores in AD. AVLT scores were higher in women versus men (P = .003). There was also a main effect of sex on HpVR and AV45 SUVR in AD. HpVR and AV45 SUVR were higher in women versus men (P < .01). There was no main effect of sex on FDG SUVR. There was a main effect of APOE-ε4 on HpVR in AD. HpVR was lower in APOE-ε4 carriers versus noncarriers (P = .004). There were no main effects of APOE-ε4 on other markers.
When excluding APOE-ε2 carriers, results were unchanged in the NC and AD groups. In contrast to the primary results, the sex by APOE interaction was significant for FDG SUVR (B = 0.05, SE = 0.03, P = .05) in MCI. APOE-ε4 was associated with lower FDG SUVR in MCI women (B = −0.09, SE = 0.02, P < .001) but not in MCI men (B = −0.03, SE = 0.02, P = .13). In addition, there was a significant sex by APOE interaction for AV45 SUVR in MCI (B = −0.11, 0.06, P = .04). The APOE-ε4 and AV45 SUVR association was stronger in MCI women (B = 0.26, SE = 0.04, P < .001) versus MCI men (B = 0.16, SE = 0.04, P < .001).
4. Discussion
Broadly, our findings support evidence of sex differences in the adverse effect of APOE-ε4 on AD-related markers. We extend previous findings by performing the following: (1) comparing these sex differences across biomarker and clinical outcomes; (2) identifying changes in sex differences that occur across disease stages; and (3) by defining MCI using an algorithmic, neuropsychological diagnostic criteria that have shown better prognostic value in terms of progression to dementia than more conventional criteria. Our novel findings are that sex differences in APOE-ε4 effects on AD-related markers varied by marker and disease stage and that the effect of APOE-ε4 on brain structure/function manifested at a later disease stage in women versus men.
Consistent with previous findings, we found that AV45 SUVR was higher in NC women versus NC men regardless of APOE genotype. We also replicated the well-evidenced finding that women outperform men in verbal memory and indicated that this difference is regardless of APOE genotype. The dichotomy of better verbal memory performance and greater Aβ pathology in NC women versus NC men suggests that women may be better equipped to compensate for disease pathology and maintain normal verbal memory performance. Consistent with some past reports [37], [38], [39], [40], we also found that NC women had a larger HpVR and a higher FDG SUVR than NC men possibly because of hormonal factors associated with female sex, such as the organization and activation effects of estrogen on the brain. A larger and a more efficient neural network in the brain region that regulates verbal memory may serve as a “brain reserve” [41], [42] that allows for compensation of the greater Aβ deposition in women and supports the female advantage in verbal memory. Inconsistent with hypotheses, sex by APOE interactions on HpVR and FDG PET in the NC group indicated an association between APOE-ε4 and lower HpVR and FDG SUVR in men only. The association between APOE and brain structure/function in healthy aging has been controversial with some, but not all, studies reporting an association [19], [43], [44], [45], [46], [47]. Neglecting APOE by sex interactive effects on brain structure/function could be a contributing factor to discordant results.
The sex differences in FDG SUVR, Aβ deposition, and HpVR (after adjusting for covariates) observed in NC were absent in MCI; however, women were able to sustain their verbal memory advantage over men in MCI despite poorer AVLT scores in MCI versus NC. In contrast to the NC group, the adverse effects of APOE-ε4 on HpVR and FDG SUVR were observed in MCI women in addition to MCI men. Findings suggest that whereas APOE-ɛ4 effects on brain structure/function manifest in men during normal cognitive aging or preclinical AD, effects do not manifest in women until MCI. Perhaps, in the MCI stage, the pathophysiological brain changes have reached a threshold of severity past which compensatory mechanisms are overwhelmed in women and the adverse effect of APOE-ε4 on verbal memory is exposed. Conversely, the adverse effect of APOE-ε4 on higher amyloid deposition was observed across sex in both NC and MCI possibly because of the hypothesized temporal evolution of AD biomarkers whereby Aβ deposition occurs before changes in brain structure and functional changes and up to a decade before clinical deficits [48].
In AD, overall sex differences included a larger HpVR and the continued female advantage in verbal memory advantage in women versus men although the sex difference in AVLT scores was much smaller than that observed in the NC and MCI groups suggesting that the advantage is attenuated with increasing disease severity. APOE-ε4 related to smaller HpVR and greater Aβ deposition but not FDG SUVR or verbal memory in AD suggesting that the effects of the disease at this advanced stage overwhelm APOE-ε4 effects on brain function and verbal memory. There was a sex by APOE interaction on AV45 SUVR in AD whereby APOE-ε4 related to AV45 SUVR in men only. As shown in Figs. 2C and 3C, the manifestation of this interaction in AD but not in MCI resulted from greater Aβ deposition in female APOE-ε4 noncarriers in AD versus MCI so that Aβ deposition was similarly high in female carriers and noncarriers with AD. Conversely, male noncarriers had similar Aβ deposition levels in MCI and AD. Similarly, Tosun et al. [22] reported a sex by APOE interaction on Aβ deposition that was driven by lower Aβ deposition in male APOE-ε4 noncarriers versus carriers and similarly high levels in female carriers and noncarriers. An increase in Aβ deposition in female noncarriers in the MCI-to-AD transition requires investigation in longitudinal studies and, if found, would challenge theories that Aβ deposition plateaus before AD diagnosis in women [48].
Our results are in contrast to some previous reports of a stronger APOE effect in women versus men on AD-related outcomes in healthy aging and on cognitive function in MCI. Discrepant findings may result from multiple contributing factors. Mixed race/ethnicity groups in prior studies may have introduced bias from population stratification that can complicate interpretation of genetic data. Race differences have been reported in the effects of APOE-ε4 on cognitive decline [49]. Studies may also have differed in the age of study samples and the specific biomarkers (e.g., cortical thickness vs. ROI-based volumetry) or cognitive outcomes examined. Many studies examined a global cognitive measure, whereas we specifically examined verbal memory, a clinical marker more reflective of AD risk. The female advantage on verbal memory may influence the effect of APOE on verbal memory. Differences in diagnostic criteria might also contribute to discrepant findings. We used the Jak/Bondi criteria, an actuarial neuropsychological diagnostic approach that eliminates use of rating scales, global cognitive screens, and subjective cognitive complaints that are commonly used in conventional diagnostic methods. When comparing Jak/Bondi and conventional criteria, conventional MCI diagnostic methods resulted in about 33% false-positive and 7% false-negative error rates [28], [29], [50]. In support of the Jak/Bondi criteria, the “false-positive” subgroup performed within normal limits across the cognitive battery and showed cerebrospinal fluid biomarker levels, cortical thickness maps, and rates of progression to dementia that were comparable to a robust NC group [28], [29], [51]. The “false-negative” group showed cerebrospinal fluid biomarker levels that were comparable to the MCI group and progressed to dementia at double the rate of the overall NC sample [50]. These diagnostic errors resulting from conventional diagnoses may have skewed results of prior studies of sex by APOE interactive effects within NC or MCI.
Our study has limitations. Our cross-sectional design precluded us from comparing rates of biomarker progression and memory decline by sex and APOE-ε4 status. Our differing and, at times, small sample sizes for each marker and diagnostic group may have led to varying and limited degrees of statistical power; however, we were able to detect a significant sex by APOE interaction in the smallest sample (AD patients with AV45 SUVR data). Our statistical power was too limited for significant findings to survive a multiple comparison correction; however, we sought to minimize comparisons by conducting sex-stratified analyses only in the presence of a significant sex by APOE-ε4 interaction. The risk for type 1 error should be considered when interpreting results. Our low number of APOE-ε4 homozygotes prohibited us from examining the additive effects of APOE-ε4 (noncarriers vs. heterozygotes vs. homozygotes). Finally, ADNI participants represent a convenience sample of volunteers who are predominantly Caucasian and well educated, which limits generalizability of results.
A study strength was our examination of the sex by APOE interaction across disease stages and across clinical and biological AD-related markers. Comparing results among diagnostic groups enabled us to discern a pattern of effects whereby the adverse effect of APOE-ε4 on brain structure/function manifests in NC men but not until clinical disease stages in women. We also revealed important sex differences in AD-related markers regardless of APOE-ε4 status. The female advantage in verbal memory in conjunction with higher greater Aβ pathology in NC women versus NC men suggests that women may be better equipped to compensate for disease pathology and maintain their verbal memory advantage.
APOE-ε4 status is a common selection criterion for intervention trials in preclinical AD. Considering that men and women may differ in the strength and trajectory of APOE-ε4 effects, failure to account for sex differences in APOE-ε4 effects on study outcomes could complicate interpretation of trials. Our findings may help to explain inconsistencies in the literature regarding APOE-ε4 effects on AD-related markers, particularly during the preclinical phase, and offer insight into biological mechanisms underlying sex differences in APOE-ε4–associated AD risk.
Research in Context.
-
1.
Systemic review: Apolipoprotein E (APOE)-ε4 genotype may confer a greater risk for Alzheimer's disease (AD) in women than in men. We are the first to examine the moderating role of sex on the effects of APOE-ε4 on multiple AD-related markers at all disease stages.
-
2.
Interpretation: Sex differences in the adverse effect of APOE-ε4 varied by AD-related marker and disease stage. Effects of APOE-ε4 on hippocampal volume and brain metabolism manifested in normal control men, whereas these effects did not manifest in women until mild cognitive impairment (MCI). In AD dementia, APOE-ε4 related to greater amyloid-β burden in men only.
-
3.
Future directions: APOE-ε4 status is a common selection criterion for intervention trials in preclinical AD. Considering that the strength and trajectory of APOE-ε4 effects differ in men and women, failure to account for sex differences in APOE-ε4 effects could complicate interpretation of trial findings.
Acknowledgments
Funding: This work was supported by the NIH (grant numbers: AG049810, AG05131, and K24 AG026431). Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant: U01 AG024904) and DOD ADNI (Department of Defense award number: W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie; Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Euroimmun; F. Hoffmann-La Roche Ltd. and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.
Footnotes
Conflict of interest: M.W.B. is paid royalties from the Oxford University Press and serves as a consultant for Eisai and Novartis. The other authors report no conflicts of interest.
References
- 1.Corder E.H., Ghebremedhin E., Taylor M.G., Thal D.R., Ohm T.G., Braak H. The biphasic relationship between regional brain senile plaque and neurofibrillary tangle distributions: modification by age, sex, and APOE polymorphism. Ann N Y Acad Sci. 2004;1019:24–28. doi: 10.1196/annals.1297.005. [DOI] [PubMed] [Google Scholar]
- 2.Strittmatter W.J., Saunders A.M., Schmechel D., Pericak-Vance M., Enghild J., Salvesen G.S. Apolipoprotein E: high-avidity binding to beta-amyloid and increased frequency of type 4 allele in late-onset familial Alzheimer disease. Proc Natl Acad Sci U S A. 1993;90:1977–1981. doi: 10.1073/pnas.90.5.1977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jin M., Shepardson N., Yang T., Chen G., Walsh D., Selkoe D.J. Soluble amyloid β-protein dimers isolated from Alzheimer cortex directly induce tau hyperphosphorylation and neuritic degeneration. Proc Natl Acad Sci U S A. 2011;108:5819–5824. doi: 10.1073/pnas.1017033108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Khan W., Giampietro V., Banaschewski T., Barker G.J., Bokde A.L., Büchel C. A multi-cohort study of APOE ɛ4 and amyloid-β effects on the hippocampus in Alzheimer's disease. J Alzheimers Dis. 2017;56:1159–1174. doi: 10.3233/JAD-161097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kuo Y.M., Emmerling M.R., Vigo-Pelfrey C., Kasunic T.C., Kirkpatrick J.B., Murdoch G.H. Water-soluble Abeta (N-40, N-42) oligomers in normal and Alzheimer disease brains. J Biol Chem. 1996;271:4077–4081. doi: 10.1074/jbc.271.8.4077. [DOI] [PubMed] [Google Scholar]
- 6.Bretsky P.M., Guralnik J.M., Launer L., Albert M., Seeman T.E. The role of APOE-ε4 in longitudinal cognitive decline MacArthur Studies of Successful Aging. Neurology. 2003;60:1077–1081. doi: 10.1212/01.wnl.0000055875.26908.24. [DOI] [PubMed] [Google Scholar]
- 7.Caselli R.J., Dueck A.C., Osborne D., Sabbagh M.N., Connor D.J., Ahern G.L. Longitudinal modeling of age-related memory decline and the APOE-ε4 effect. N Engl J Med. 2009;361:255–263. doi: 10.1056/NEJMoa0809437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tomic J.L., Pensalfini A., Head E., Glabe C.G. Soluble fibrillar oligomer levels are elevated in Alzheimer's disease brain and correlate with cognitive dysfunction. Neurobiol Dis. 2009;35:352–358. doi: 10.1016/j.nbd.2009.05.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lupton M.K., Strike L., Hansell N.K., Wen W., Mather K.A., Armstrong N.J. The effect of increased genetic risk for Alzheimer's disease on hippocampal and amygdala volume. Neurobiol Aging. 2016;40:68–77. doi: 10.1016/j.neurobiolaging.2015.12.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Smith G.E., Bohac D.L., Waring S.C., Kokmen E., Tangalos E.G., Ivnik R.J. Apolipoprotein E genotype influences cognitive `phenotype' in patients with Alzheimer's disease but not in healthy control subjects. Neurology. 1998;50:355–362. doi: 10.1212/wnl.50.2.355. [DOI] [PubMed] [Google Scholar]
- 11.Landau S.M., Harvey D., Madison C.M., Koeppe R.A., Reiman E.M., Foster N.L. Associations between cognitive, functional, and FDG-PET measures of decline in AD and MCI. Neurobiol Aging. 2011;32:1207–1218. doi: 10.1016/j.neurobiolaging.2009.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tierney M.C., Szalai J.P., Snow W.G., Fisher R.H., Tsuda T., Chi H. A prospective study of the clinical utility of APOE genotype in the prediction of outcome in patients with memory impairment. Neurology. 1996;46:149–154. doi: 10.1212/wnl.46.1.149. [DOI] [PubMed] [Google Scholar]
- 13.Bretsky P.M., Buckwalter J.G., Seeman T.E., Miller C.A., Poirier J., Schellenberg G.D. Evidence for an interaction between apolipoprotein E genotype, gender, and Alzheimer disease. Alzheimer Dis Assoc Disord. 1999;13:216–221. doi: 10.1097/00002093-199910000-00007. [DOI] [PubMed] [Google Scholar]
- 14.Payami H., Montee K.R., Kaye J.A., Bird T.D., Yu C.E., Wijsman E.M. Alzheimer's disease, apolipoprotein E4, and gender. JAMA. 1994;271:1316–1317. [PubMed] [Google Scholar]
- 15.Poirier J., Bertrand P., Kogan S., Gauthier S., Davignon J., Bouthillier D. Apolipoprotein E polymorphism and Alzheimer's disease. Lancet. 1993;342:697–699. doi: 10.1016/0140-6736(93)91705-q. [DOI] [PubMed] [Google Scholar]
- 16.Altmann A., Tian L., Henderson V.W., Greicius M.D. Sex modifies the APOE-related risk of developing Alzheimer disease. Ann Neurol. 2014;75:563–573. doi: 10.1002/ana.24135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Neu S.C., Pa J., Kukull W., Beekly D., Kuzma A., Gangadharan P. Apolipoprotein E genotype and sex risk factors for Alzheimer disease: a meta-analysis. JAMA Neurol. 2017;74:1178–1189. doi: 10.1001/jamaneurol.2017.2188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Damoiseaux J.S., Seeley W.W., Zhou J., Shirer W.R., Coppola G., Karydas A. Gender modulates the APOE-ε4effect in healthy older adults: convergent evidence from functional brain connectivity and spinal fluid tau levels. J Neurosci. 2012;32:8254–8262. doi: 10.1523/JNEUROSCI.0305-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Sampedro F., Vilaplana E., de Leon M.J., Alcolea D., Pegueroles J., Montal V. APOE-by-sex interactions on brain structure and metabolism in healthy elderly controls. Oncotarget. 2015;6:26663–26674. doi: 10.18632/oncotarget.5185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jack C.R., Jr., Wiste H.J., Weigand S.D., Knopman D.S., Vemuri P., Mielke M.M. Age, sex, and APOE-ε4effects on memory, brain structure, and β-amyloid across the adult life span. JAMA Neurol. 2015;72:511–519. doi: 10.1001/jamaneurol.2014.4821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Fleisher A., Grundman M., Jack C.R., Petersen R.C., Taylor C., Kim H.T. Sex, apolipoprotein E ε4 status, and hippocampal volume in mild cognitive impairment. Arch Neurol. 2005;62:953–957. doi: 10.1001/archneur.62.6.953. [DOI] [PubMed] [Google Scholar]
- 22.Tosun D., Insel P.S., Weiner M.W. APOE-ɛ4 genotype by gender interactions in regional amyloid accumulation in the Alzheimer's disease continuum. Alzheimers Dement. 2015;11:P195. [Google Scholar]
- 23.Hyman B.T., Gomez-Isla T., Briggs M., Chung H., Nichols S., Kohout F. Apolipoprotein E and cognitive change in an elderly population. Ann Neurol. 1996;40:55–66. doi: 10.1002/ana.410400111. [DOI] [PubMed] [Google Scholar]
- 24.Mortensen E.L., Hogh P. A gender difference in the association between APOE genotype and age-related cognitive decline. Neurology. 2001;57:89–95. doi: 10.1212/wnl.57.1.89. [DOI] [PubMed] [Google Scholar]
- 25.Petersen R.C., Smith G.E., Waring S.C., Ivnik R.J., Tangalos E.G., Kokmen E. Mild cognitive impairment: clinical characterisation and outcome. Arch Neurol. 1999;56:303–308. doi: 10.1001/archneur.56.3.303. [DOI] [PubMed] [Google Scholar]
- 26.Jack C.R., Knopma D.S., Jagust W.J., Shaw L.M., Aisen P.S., Weiner M.W. Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade. Lancet Neurol. 2010;9:119–128. doi: 10.1016/S1474-4422(09)70299-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Jack C.R., Jr., Vemuri P., Wiste H.J., Weigand S.D., Aisen P.S., Trojanowski J.Q., Alzheimer's Disease Neuroimaging Initiative Evidence for ordering of Alzheimer disease biomarkers. Arch Neurol. 2011;68:1526–1535. doi: 10.1001/archneurol.2011.183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Bondi M.W., Edmonds E.C., Jak A.J., Clark L.R., Delano-Wood L., McDonald C.R. Neuropsychological criteria for mild cognitive impairment improves diagnostic precision, biomarker associations, and progression rates. J Alzheimers Dis. 2014;42:275–289. doi: 10.3233/JAD-140276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Jak A.J., Preis S.R., Beiser A.S., Seshadri S., Wolf P.A., Bondi M.W. Neuropsychological criteria for mild cognitive impairment and dementia risk in the Framingham Heart Study. J Int Neuropsychol Soc. 2016;22:937–943. doi: 10.1017/S1355617716000199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Schmidt M. Western Psychological Services; California: 1996. Rey Auditory Verbal Learning Test: A Handbook. [Google Scholar]
- 31.Bondi M.W., Salmon D.P., Galasko D., Thomas R.G., Thal L.J. Neuropsychological function and apolipoprotein E genotype in the preclinical detection of Alzheimer's disease. Psychol Aging. 1999;14:295–303. doi: 10.1037//0882-7974.14.2.295. [DOI] [PubMed] [Google Scholar]
- 32.Grober E., Kawas C. Learning and retention in preclinical and early Alzheimer's disease. Psychol Aging. 1997;12:183–188. doi: 10.1037//0882-7974.12.1.183. [DOI] [PubMed] [Google Scholar]
- 33.Jack C.R., Jr., Bernstein M.A., Fox N.C., Thompson P., Alexander G., Harvey D. The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging. 2008;27:685–691. doi: 10.1002/jmri.21049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hsu Y.Y., Schuff N., Du A.T., Mark K., Zhu X., Hardin D. Comparison of automated and manual MRI volumetry of hippocampus in normal aging and dementia. J Magn Reson Imaging. 2002;16:305–310. doi: 10.1002/jmri.10163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Jagust W.J., Bandy D., Chen K., Foster N.L., Landau S.M., Mathis C.A. The Alzheimer's Disease Neuroimaging Initiative positron emission tomography core. Alzheimers Dement. 2010;6:221–229. doi: 10.1016/j.jalz.2010.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.McKhann G., Drachman D., Folstein M., Katzman R., Price D., Stadlan E.M. Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease. Neurology. 1984;34:939–944. doi: 10.1212/wnl.34.7.939. [DOI] [PubMed] [Google Scholar]
- 37.Sundermann E.E., Biegon A., Rubin L.H., Lipton R.B., Mowrey W., Landau S. Better verbal memory in women than men in MCI despite similar levels of hippocampal atrophy. Neurology. 2016;86:1368–1376. doi: 10.1212/WNL.0000000000002570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Sundermann E.E., Maki P.M., Rubin L.H., Lipton R.B., Landau S., Biegon A. Female advantage in verbal memory: evidence of sex-specific cognitive reserve. Neurology. 2016;87:1916–1924. doi: 10.1212/WNL.0000000000003288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Andreason P.J., Zametkin A.J., Guo A.C., Baldwin P., Cohen R.M. Gender-related differences in regional cerebral glucose metabolism in normal volunteers. Psychiatry Res. 1994;51:175–183. doi: 10.1016/0165-1781(94)90037-x. [DOI] [PubMed] [Google Scholar]
- 40.Filipek P.A., Richelme C., Kennedy D.N., Caviness V.S., Jr. The young adult human brain: an MRI-based morphometric analysis. Cereb Cortex. 1994;4:344–360. doi: 10.1093/cercor/4.4.344. [DOI] [PubMed] [Google Scholar]
- 41.Stern Y. What is cognitive reserve? Theory and research application of the reserve concept. J Int Neuropsychol Soc. 2002;8:448–460. [PubMed] [Google Scholar]
- 42.Stern Y., Zarahn E., Hilton H.J., Flynn J., DeLaPaz R., Rakitin B. Exploring the neural basis of cognitive reserve. J Clin Exp Neuropsychol. 2003;25:691–701. doi: 10.1076/jcen.25.5.691.14573. [DOI] [PubMed] [Google Scholar]
- 43.Liu Y., Yu J.T., Wang H.F., Han P.R., Tan C.C., Wang C. APOE genotype and neuroimaging markers of Alzheimer's disease: systematic review and meta-analysis. J Neurol Neurosurg Psychiatry. 2015;86:127–134. doi: 10.1136/jnnp-2014-307719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Cherbuin N., Leach L.S., Christensen H., Anstey K.J. Neuroimaging and APOE genotype: a systematic qualitative review. Dement Geriatr Cogn Disord. 2007;24:348–362. doi: 10.1159/000109150. [DOI] [PubMed] [Google Scholar]
- 45.Reiman E.M., Chen K., Alexander G.E., Caselli R.J., Bandy D., Osborne D. Correlations between apolipoprotein E epsilon4 gene dose and brain-imaging measurements of regional hypometabolism. Proc Natl Acad Sci U S A. 2005;102:8299–8302. doi: 10.1073/pnas.0500579102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Corder E.H., Jelic V., Basun H., Lannfelt L., Valind S., Winblad B. No difference in cerebral glucose metabolism in patients with Alzheimer disease and differing apolipoprotein E genotypes. Arch Neurol. 1997;54:273–277. doi: 10.1001/archneur.1997.00550150035013. [DOI] [PubMed] [Google Scholar]
- 47.Samuraki M., Matsunari I., Chen W.P., Shima K., Yanase D., Takeda N. Glucose metabolism and gray-matter concentration in apolipoprotein E ε4 positive normal subjects. Neurobiol Aging. 2012;33:2321–2323. doi: 10.1016/j.neurobiolaging.2011.11.020. [DOI] [PubMed] [Google Scholar]
- 48.Jack C.R., Jr., Wiste H.J., Lesnick T.G., Weigand S.D., Knopman D.S., Vemuri P. Brain β-amyloid load approaches a plateau. Neurology. 2013;80:890–896. doi: 10.1212/WNL.0b013e3182840bbe. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Barnes L.L., Arvanitakis Z., Yu L., Kelly J., de Jager P.L., Bennett D.A. Apolipoprotein E and change in episodic memory in blacks and whites. Neuroepidemiology. 2013;40:211–219. doi: 10.1159/000342778. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Edmonds E.C., Delano-Wood L., Jak A.J., Galasko D.R., Salmon D.P., Bondi M.W. “Missed” mild cognitive impairment: high false-negative error rate based on conventional diagnostic criteria. J Alzheimers Dis. 2016;52:685–691. doi: 10.3233/JAD-150986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Edmonds E.C., Eppig J., Bondi M.W., Leyden K.M., Goodwin B., Delano-Wood L. Heterogeneous cortical atrophy patterns in MCI not captured by conventional diagnostic criteria. Neurology. 2016;87:2108–2116. doi: 10.1212/WNL.0000000000003326. [DOI] [PMC free article] [PubMed] [Google Scholar]