Skip to main content
Alzheimer's & Dementia : Diagnosis, Assessment & Disease Monitoring logoLink to Alzheimer's & Dementia : Diagnosis, Assessment & Disease Monitoring
. 2016 May 6;3:73–82. doi: 10.1016/j.dadm.2016.02.008

Plasma phospholipids and prevalence of mild cognitive impairment and/or dementia in the ARIC Neurocognitive Study (ARIC-NCS)

Danni Li a,, Jeffrey R Misialek b, Eric Boerwinkle c, Rebecca F Gottesman d, A Richey Sharrett e, Thomas H Mosley f, Josef Coresh e, Lisa M Wruck g, David S Knopman h, Alvaro Alonso b
PMCID: PMC4925799  PMID: 27408938

Abstract

Introduction

Phospholipids are altered in brains of patients with dementia and some studies suggest their plasma levels may be useful in the detection of mild cognitive impairment (MCI) and dementia.

Methods

We measured 188 plasma metabolites in participants who underwent a detailed neuropsychological assessment and classified as normal (n = 153), MCI (n = 145), or dementia (n = 143) by expert adjudication.

Results

Among 10 phospholipids recently implicated as altered in dementia, higher concentration of PC aa C36:6 was significantly associated with decreased prevalence of dementia (odds ratio = 0.71, 95% confidence interval = 0.50–1.00 per 1−SD increase). Adding these phospholipids to a model including multiple predictors of dementia led to only minimal improvement in detection (C statistic changed from 0.702 to 0.71).

Discussion

Some phospholipids and metabolites were altered in MCI and dementia but cross-sectional association was relatively weak and did not improve detection of MCI and dementia beyond information provided by clinical variables.

Keywords: Phospholipids, Plasma, Mild cognitive impairment, MCI, Dementia, Metabolomics, Metabolites, ARIC-NCS, ARIC, Alzheimer's disease, AD

1. Introduction

There is a growing interest in discovery of plasma biomarkers for prediction and diagnosis of mild cognitive impairment (MCI) and dementia, in particular MCI and dementia due to Alzheimer's disease (AD). Currently, amnestic MCI (aMCI) and AD-type dementia, similar to other MCI and dementia, are generally diagnosed based on a full assessment of cognitive function, a neurological examination, and clinical examination of history in cognitive and behavioral changes. Positron emission tomography neuroimaging and cerebrospinal fluid proteins are important but invasive and expensive tools for verifying the diagnosis. Owing to the complexity of AD pathology, mounting evidence shows that no single biomarker can yield enough sensitivity and specificity, and multiple biomarkers may be necessary to diagnose AD and track its progression. Plasma biomarkers are less invasive and may be used as a first-step screen of AD. Moreover, the less invasive nature of plasma biomarkers makes them potentially more suitable for monitoring disease progression and treatment response to potential therapies in AD.

Previous work has documented altered concentrations of phospholipids in brains of aging, cognitive function decline, MCI, and dementia [1], [2], [3]. Furthermore, plasma phospholipids were associated with cognitive function during middle adulthood [4], the risk of decline in verbal fluency [5], and a significant reduction in risk of developing all-cause dementia [6]. Recently, Mapstone et al. reported a panel of plasma phospholipids that identified cognitive normal adults who would progress to either aMCI or AD within 2–3 years from those who remained cognitively normal (the area under curve [AUC] of the receiver-operating characteristic [ROC] analysis was 0.96 [95% confidence interval, 0.93–0.99]) [7]. In addition to the index finding, these 10 phospholipids also discriminated aMCI/AD from cognitive normal with an AUC of 0.827 in the initial discovery (n = 88) and an AUC of 0.77 in an independent validation (n = 41) [7]. Subsequently, three publications suggested that plasma phospholipids and metabolites might aid in the clinical diagnosis of MCI and dementia [8], [9], [10]. With the general aim of validating the biomarkers identified by Mapstone et al. for detection of MCI and dementia and not for prediction, we examined the cross-sectional association of concentrations of selected phospholipids with the prevalence of MCI or dementia in a subset of participants in the Atherosclerosis Risk in Communities (ARIC) Neurocognitive Study (ARIC-NCS).

2. Methods

2.1. ARIC-NCS study

The ARIC study is a prospective cohort study investigating the etiology of atherosclerotic diseases in a middle-aged, predominantly biracial population. A detailed study design description was published [11].

2.2. Standard protocol approvals and patient consents

The ARIC Study and ARIC-NSC protocols were approved by the institutional review board of each participating center, and informed consent was obtained from participants at each study visit.

2.3. Measures of cognitive performance

As part of the ARIC-NCS, all participants underwent a physical examination, collection of blood samples, and a detailed neurocognitive assessment in 2011–2013. Measures of cognitive function in ARIC-NCS have been described in detail [12].

Participants who had a low mini mental state examination (MMSE) score (<21 in whites, <19 in African Americans), those with low scores in the cognitive testing (<1.5 standard deviations below norms applicable to our specific populations in any single domain) and cognitive decline (as evidenced by decrease in cognitive tests from previous examinations), together with a random subset of participants without evidence of cognitive impairment, underwent a neurological examination, and were administered the Clinical Dementia Rating (CDR) scale, both the informant and the subject interviews. A subset of participants without contraindications underwent a brain magnetic resonance imaging (MRI) study, as described elsewhere [13]. Briefly, MRI scans were provided in each site on 3-Tesla scanners using a common set of sequences including 3-D volumetric magnetization and fluid-attenuated inversion recovery sequences.

2.4. Diagnosis and adjudication of MCI and dementia

Two study reviewers, a neurologist and a neuropsychologist, reviewed independently data on each participant and rendered a diagnosis of normal cognition, MCI, or dementia. After the established of the syndromic diagnosis, an etiologic diagnosis was made for participants with MCI or dementia diagnoses. Discordant cases were assigned to a third independent reviewer [13].

Functional assessment questionnaire (FAQ) and CDR were used. Although the FAQ score was not administrated as a distinct scale, the items for the FAQ were embedded with the CDR. A diagnosis of MCI was assigned in persons without dementia who met the three criteria below: (1) FAQ ≤5 or CDR sum of boxes ≤3, (2) at least one neuropsychological cognitive domain Z score <−1.5 or clock time perception reading failure, and (3) documented decline in ARIC serial cognitive test battery of these tests: delayed word recall (DWR), digit symbol substitution (DSS), and word fluency (WF) (i.e., falling at or below the worst 20th percentile of change on >1 test or below the worst 10th percentile on at least 1 test; with change calculated as current score minus the highest prior score). Dementia diagnosis was made either (1) by a low MMSE score (<21 in whites, <19 in African Americans), even in the absence of more complete cognitive testing, if, in the judgment of the Classification Committee, any prior DWR, DSS, and WF scores were not indicative of dementia, or (2) by meeting all three of the following criteria: FAQ ≥ 5 or CDR sum of boxes >3, at least two neuropsychiatric cognitive domain scores < −1.5, and documented decline in ARIC serial cognitive test battery (as defined above).

Etiologic diagnoses were recorded as primary or secondary. Primary diagnoses and all vascular diagnoses, whether primary or secondary, were adjudicated. The diagnosis of AD as an etiologic diagnosis of MCI or dementia as a primary diagnosis was a clinical one and was based on the presence of cognitive syndrome that was not of abrupt onset and included memory impairment, in the absence of features of other specific diagnoses sufficient to cause the cognitive impairment. Cerebrovascular disease–related MCI and/or dementia was recorded if there was (1) history of stroke temporally related to an abrupt onset of the cognitive disorder, (2) bilateral or multiple infarcts or extensive white-matter hyperintensities on imaging, or (3) physical examination evidence of a typical stroke pattern. In this study, AD-type dementia includes dementia with AD as the primary diagnosis with or without a secondary diagnosis; all dementia cases that are not AD-type dementia are considered as non-AD type dementia.

2.5. Sample collection, handling, and storage

The ARIC-NCS study collected fasting blood samples from all participants using standard protocols. Fasting EDTA whole blood was collected and placed in ice bath until further processing by centrifugation (i.e., 10 minutes at 3000 g in 4°C). After the centrifugation, plasma supernatant was aliquoted and stored in −80°C freezers until further shipping on dry ice for long-term storage. The sample collection and processing protocols are compatible with stability of plasma lipoproteins [14] and phospholipids [15]. EDTA plasma samples that were never thawed were used in this study for targeted metabolomics analysis to minimize the effects of freeze-thaw on plasma phospholipids and metabolites.

2.6. Sample selection

The ARIC-NCS had 216 dementia, 934 MCI, and 1547 cognitive normal black and white participants. We sampled approximately 73 cases from each MCI/race group and dementia/group without regard to etiologic diagnoses. Then, we sampled approximately 73 controls from each race group by frequency matching on APOE genotype, age, sex, race, education, and study center (in order) with the frequency distribution of controls matching that of the combined MCI and dementia cases.

2.7. Phospholipids measurement

Targeted metabolomics analysis of plasma samples was performed using the Biocrates Absolute-IDQ P180 kits (Biocrates; Life Science AG, Innsbruck, Austria). The plasma samples were processed as per the manufacturer instructions and analyzed on a triple-quadrupole mass spectrometer (AB Sciex QTRAP 6500). As part of the quality control, three concentrations of quality controls were included in each kit to monitor imprecisions (% coefficient variances [CVs]) of measuring these 188 metabolites. Means, standard deviations (SDs), and CVs of all the 188 metabolites in seven kits (the number of kits needed for analysis of the 441 samples) were calculated, and imprecisions >80% of the metabolites were <20% for all three concentrations of QCs (Supplementary Table 1). Data analysis was performed using the MetIQ software (Biocrates).

2.8. Statistical analysis

Metabolite concentrations were log transformed and modeled in standard deviation units when possible. Eleven metabolites had some 0 values, to which we imputed the lowest nonzero measurement for analysis. Overall, 28 metabolites had >80% missing or below limit of detection (LOD) or had a failure in model convergence and therefore were excluded (Supplementary Table 2). Ten metabolites with 50%–80% missing or below LOD values were categorized into three groups: <LOD, ≥LOD to < median, and ≥median (Supplementary Table 2). The main analysis focused on 9 of the 10 phospholipids identified in the previous publication by Mapstone et al. as predictors of conversion from normal to cognitive impairment (the 10th metabolite, C16:1-OH, had values below the LOD in all measurements) [7]. In the primary analysis, a multinomial logistic regression was used to assess the association of the individual metabolites with the prevalence of MCI and dementia (three-level dependent variable: normal, MCI, dementia, n = 441). The following variables were used for adjustment: model 1: age, race, sex, and APOE genotype and model 2: model 1 + educational level, diabetes mellitus, body mass index, drinking status, smoking status, sports index, systolic blood pressure, use of antihypertensive medications, prevalent coronary heart disease, prevalent heart failure, prevalent stroke, total cholesterol, high-density lipoprotein cholesterol, and triglycerides at the ARIC-NCS. These covariates were assessed at ARIC visit 5/NCS examination, the same time when the plasma samples were taken from the 441 participants. Secondary separate analyses were performed based on etiological groups of discriminating aMCI and AD type dementia from normal (n = 245). To measure the ability of this panel of 10 phospholipids in the classification of normal versus MCI and/or dementia, we calculated the C statistic from binary logistic regression models with dementia and/or MCI (vs. normal) as the outcome with individual phospholipids modeled as a 1−SD difference in the log-transformed phospholipid (unless noted). The C statistic is the proportion of pairs of subjects with the outcome and subjects without the outcome in which the subject who experienced the outcome had a higher predicted probability of the outcome than the subject without the outcome [16]. This statistic corresponds to the AUC and is part of the standard output of PROC LOGISTIC in SAS. Finally, linear regression analysis was performed to estimate the association of concentrations of these nine phospholipids with neuropsychological tests, modeled as z scores. We used SAS, v 9.3 (SAS Inc., Cary, North Carolina), for the statistical analysis.

In an exploratory, hypothesis-generating analysis, we repeated the analyses for the rest of 151 metabolites assessed, which excluded the 28 metabolites with >80% values lower than LOD and nine phospholipids from the main analysis. This hypothesis-generating analysis adjusted for the same variables used in model 2 above and was corrected for multiple comparisons using a Bonferroni correction [17]. Therefore, only associations with a P value < .00033 (0.05/151 metabolites) were considered statistically significant.

3. Results

3.1. Baseline characteristics of individuals adjudicated as normal, MCI, and dementia in ARIC

Table 1 presents the characteristics of the 441 ARIC-NCS participants by dementia status in this study. Overall, individuals with dementia or MCI were older had a worse cardiovascular risk profile and a higher prevalence of APOE ε4 allele.

Table 1.

Baseline characteristics by dementia status, ARIC-NCS study, 2011–2013

Characteristics Normal (n = 153) Mild cognitive impairment (n = 145) Dementia (n = 143)
Age, years 77.6 (5.5) 76.5 (5.6) 79.7 (5.1)
African American, % 30.1 17.2 37.8
Female, % 60.1 47.6 55.2
Body mass index, kg/m2 28.6 (5.5) 29.7 (6.0) 27.9 (5.7)
Current drinker, % 44.4 44.8 31.5
Current smoker, % 3.9 4.8 4.2
Diabetes, % 30.1 46.2 39.2
High school graduate, % 45.8 45.5 32.2
HDL cholesterol, mg/dL 54.4 (14.2) 49.1 (12.9) 52.5 (15.8)
Hypertension medication, % 77.8 79.3 80.4
Prevalent CHD, % 9.8 20.0 25.9
Prevalent HF, % 1.3 2.1 7.0
Prevalent stroke, % 1.3 5.5 10.5
Systolic blood pressure, mmHg 131.1 (18.5) 130.7 (17.7) 134.3 (21.3)
Sports index 2.6 (0.8) 2.6 (0.8) 2.3 (0.7)
Total cholesterol, mg/dL 184.6 (39.8) 174.6 (41.6) 183.9 (47.4)
Triglycerides, mg/dL 120.9 (60.9) 125.6 (63.3) 127.3 (76.7)
APOE, % 44 1.3 2.1 9.1
24 or 34 26.1 33.8 37.8
Other 69.9 62.1 48.3
Data missing 2.6 2.1 4.9

3.2. Associations of plasma phospholipids/metabolites with MCI and/or dementia

Our main analysis focused on the 10 phospholipids identified by Mapstone et al. [7]. Our hypothesis, based on the findings of Mapstone et al., was that higher concentrations of these phospholipids would be associated with a lower prevalence of MCI and/or dementia. Of these 10 metabolites, concentrations of C16:1-OH in all participants were lower than LOD (and considered unreliable). Therefore, we excluded C16:1-OH from the analysis. Table 2 illustrates results from multinomial logistic models estimating the association of the remaining nine phospholipids with MCI and/or dementia. In unadjusted and minimally adjusted models, higher concentrations of PC aa C40:2 were significantly associated with a lower prevalence of MCI, whereas higher concentrations of PC aa C36:6 were significantly associated with lower prevalence of dementia, consistent with our hypothesis. After adjusting for additional covariates in model 2, only the concentration of PC aa C36:6 remained significantly associated with dementia prevalence (odds ratio = 0.71, 95% confidence interval, 0.50–1.00).

Table 2.

Odds ratio (OR) and 95% confidence interval (CI) of MCI and dementia by phospholipids (per 1−SD difference in log-transformed metabolite, unless noted) from a multinomial logistic regression model, ARIC-NCS study, 2011–2013

Phospholipids n MCI vs. normal OR (95% CI) Dementia vs. normal OR (95% CI) MCI & dementia vs. normal OR (95% CI)
PC aa C36:6
 Model 0 441 0.85 (0.67–1.07) 0.68 (0.54–0.87) 0.77 (0.62–0.93)
 Model 1 441 0.81 (0.62–1.05) 0.69 (0.53–0.91) 0.76 (0.60–0.94)
 Model 2 441 1.01 (0.73–1.40) 0.71 (0.50–1.00) 0.86 (0.64–1.13)
PC aa C38:0
 Model 0 441 0.82 (0.65–1.04) 0.93 (0.74–1.17) 0.87 (0.72–1.06)
 Model 1 441 0.95 (0.74–1.21) 0.88 (0.68–1.15) 0.92 (0.74–1.13)
 Model 2 441 1.14 (0.85–1.53) 0.93 (0.68–1.27) 1.04 (0.81–1.34)
PC aa C38:6
 Model 0 441 0.86 (0.68–1.08) 0.83 (0.66–1.04) 0.84 (0.69–1.03)
 Model 1 441 0.90 (0.71–1.14) 0.80 (0.62–1.03) 0.85 (0.69–1.05)
 Model 2 441 1.08 (0.81–1.43) 0.82 (0.60–1.11) 0.96 (0.75–1.24)
PC aa C40:1
 Model 0 441 0.76 (0.55–1.04) 0.95 (0.71–1.29) 0.85 (0.66–1.11)
 Model 1 441 0.83 (0.59–1.15) 0.94 (0.68–1.30) 0.88 (0.67–1.16)
 Model 2 441 0.94 (0.65–1.38) 0.96 (0.65–1.42) 0.94 (0.68–1.31)
PC aa C40:2
 Model 0 441 0.73 (0.58–0.92) 1.01 (0.80–1.27) 0.86 (0.70–1.05)
 Model 1 441 0.77 (0.60–0.98) 0.97 (0.76–1.24) 0.86 (0.70–1.06)
 Model 2 441 0.85 (0.63–1.15) 1.04 (0.75–1.43) 0.92 (0.71–1.20)
PC aa C40:6
 Model 0 441 0.85 (0.67–1.07) 0.83 (0.66–1.04) 0.84 (0.68–1.02)
 Model 1 441 0.91 (0.71–1.16) 0.79 (0.61–1.02) 0.85 (0.69–1.06)
 Model 2 441 1.04 (0.78–1.38) 0.75 (0.55–1.03) 0.91 (0.71–1.17)
PC ae C40:6
 Model 0 441 0.81 (0.64–1.01) 0.89 (0.71–1.12) 0.85 (0.70–1.03)
 Model 1 441 0.89 (0.70–1.14) 0.84 (0.65–1.08) 0.88 (0.71–1.08)
 Model 2 441 1.12 (0.83–1.51) 0.86 (0.62–1.20) 1.02 (0.78–1.33)
Lyso PC a C18:2
 Model 0 441 1.03 (0.82–1.29) 1.15 (0.91–1.45) 1.09 (0.89–1.32)
 Model 1 441 0.93 (0.73–1.19) 1.14 (0.89–1.46) 1.04 (0.85–1.28)
 Model 2 441 1.07 (0.81–1.41) 1.15 (0.85–1.54) 1.12 (0.88–1.43)
Propionyl-L-carnitine (C3)
 Model 0 440 1.17 (0.93–1.47) 1.08 (0.86–1.36) 1.12 (0.92–1.37)
 Model 1 440 1.14 (0.90–1.45) 1.13 (0.89–1.45) 1.13 (0.92–1.40)
 Model 2 440 1.04 (0.78–1.38) 1.02 (0.76–1.37) 1.04 (0.82–1.34)

NOTE. Model 0: unadjusted logistic regression. Model 1: Logistic regression adjusted for age, race, sex, & APOE. Model 2: Model 1 with additional adjustment for APOE, body mass index, diabetes mellitus, drinking status, educational level, HDL cholesterol, smoking status, sports index, systolic blood pressure, total cholesterol, triglycerides, use of antihypertensive medications, and prevalent coronary heart disease, heart failure, or stroke at baseline.

OR per 1−SD change in the log(phospholipid).

OR for phospholipid as a 3 group ordinal variable (below LOD, ≥LOD− < Median, ≥Median).

Table 3 shows the ability (C-statistic) of these nine phospholipids in classifying individuals as cognitive normal versus MCI and/or dementia. Individual phospholipids had C-statistics in the range of 0.522–0.580 in the unadjusted model. When combined, these phospholipids had a slightly improved C-statistic of 0.602. A model combining all the covariates but not concentrations of phospholipids achieved a C-statistic of 0.702. Adding to this model, individual phospholipids or all nine phospholipids simultaneously led to minimal improvements in the C-statistic (range, 0.702–0.713). Furthermore, we performed the analyses by focusing on MCI and dementia with AD as the primary diagnosis with or without a secondary diagnosis. None of these nine phospholipids were significantly associated with AD-type MCI or dementia (Supplementary Table 3). Similarly, these phospholipids did not provide added value in the detection of AD-type MCI or dementia prevalence (Supplementary Table 4).

Table 3.

C-statistic from binary logistic regression models with dementia/MCI (vs normal) as the outcome and individual phospholipids modeled as a 1-SD difference in the log-transformed phospholipid unless noted, ARIC-NCS study, 2011–2013

Models C statistic
Model 0 Model 2
Model (No Phospholipids) 0.702
 Model + PC aa C36:6 0.580 0.704
 Model + PC aa C38:0 0.534 0.703
 Model + PC aa C38:6 0.544 0.702
 Model + PC aa C40:1 0.522 0.702
 Model + PC aa C40:2 0.548 0.703
 Model + PC aa C40:6 0.546 0.703
 Model + PC ae C40:6 0.547 0.704
 Model + Lyso PC a C18:2 0.522 0.704
 Model + Propionyl-L-carnitine (C3) 0.532 0.701
Model + all 9 phospholipids 0.602 0.713

NOTE. Model 0: Unadjusted logistic regression. Model 2: model 1 with additional adjustment for APOE, body mass index, diabetes mellitus, drinking status, educational level, HDL cholesterol, smoking status, sports index, systolic blood pressure, total cholesterol, triglycerides, use of antihypertensive medications, and prevalent coronary heart disease, heart failure, or stroke at baseline.

The phospholipid is modeled as a three-group ordinal variable (below LOD, >LOD− <Median, >Median).

In an exploratory, hypothesis-generating analysis, we also analyzed the associations of the remaining 151 phospholipids/metabolites with MCI or dementia (Supplementary Table 5). After adjusting for covariates (in model 2) and correcting for multiple comparisons, only SM(OH) C22:1 was statistically significantly associated with dementia (P value = .0003).

3.3. Associations of plasma phospholipids/metabolites with cognitive performance

We determined the associations of the nine phospholipids with scores in the MMSE and 12 neuropsychological tests across the 441 participants in this study (153 normal, 145 MCI, and 143 dementia; Table 4). After adjusting for multiple covariates, higher concentrations of PC aa C40:6 and lyso PC aa C18:2 were associated respectively with higher and lower values in the MMSE score. Higher concentrations of PC aa C36:6, PC aa C38:0, PC aa 38:6, PC aa C40:2, PC aa 40:6, and PC ae 40:6 were significantly associated with better scores in incidental learning, a neuropsychological test assessing memory performance. Similarly, higher concentrations of PC aa C36:6, PC aa 38:0, PC ae 40:6, and PC aa C40:6 were associated with better scores in tests evaluating memory; higher concentrations of PC aa C38:0 and PC ae C40:6 were associated with better scores in tests evaluating attention; higher concentrations of PC aa C36:6 and C3 are associated with better scores in tests evaluating executive function. Furthermore, we examined the association of two of these nine phospholipids (PC aa C38:0 and PC ae C40: 6) with cognitive performance in the 153 cognitive normal participants (Table 5). We found that overall their associations with cognitive performance were stronger in cognitive normal participants than in the whole study population (Table 5 vs. Table 4).

Table 4.

The associations of selected phospholipids/metabolites with scores in the MMSE and 12 neuropsychological tests across the 441 participants in ARIC-NCS, 2011–2013

Neuropsychological tests PC aa C36:6
PC aa C38:0
PC aa C38:6
n β 95% CI P value n β 95% CI P value n β 95% CI P value
Mini-mental state examination 388 0.06 −0.04 0.16 .22 388 0.04 −0.06 0.13 .41 388 0.07 −0.03 0.16 .16
Cognitive function domain
 Memory
 Delayed word recall 415 0.12 0.01 0.23 .03 415 0.09 −0.01 0.19 .09 415 0.05 −0.05 0.14 .36
 Logical Memory Test part A 395 0.08 −0.04 0.19 .18 395 0.14 0.04 0.25 .01 395 0.10 0.0002 0.21 .05
 Logical Memory Test part B 385 0.04 −0.08 0.16 .53 385 0.15 0.04 0.25 .01 385 0.06 −0.04 0.17 0.23
 Incidental learning 378 0.23 0.11 0.35 .0002 378 0.24 0.12 0.35 <.0001 378 0.19 0.08 0.30 .001
 Language
 Animal naming 425 0.09 −0.01 0.20 .09 425 0.02 −0.08 0.12 .70 425 0.01 −0.08 0.11 .82
 Boston Naming Test 407 0.07 −0.02 0.16 .14 407 0.05 −0.03 0.14 .20 407 0.02 −0.06 0.10 .63
 Word fluency 407 0.09 −0.02 0.20 .10 407 0.02 −0.08 0.12 .66 407 0.03 −0.06 0.13 .49
 Visuospatial
 Clock time perception 405 0.10 −0.02 0.22 .10 405 0.07 −0.04 0.18 .22 405 0.05 −0.06 0.16 .39
 Attention
 Trail making test part A 379 0.07 −0.03 0.18 .16 379 0.01 −0.09 0.10 .89 379 0.02 −0.08 0.11 .74
 Digit span backward 402 0.02 −0.09 0.13 .75 402 0.12 0.01 0.22 .03 402 0.06 −0.04 0.16 .24
 Executive function
 Digit symbol substitution 389 0.11 0.01 0.21 .03 389 −0.005 −0.10 0.09 .92 389 0.0001 −0.09 0.09 .99
 Trail making test part B 321 0.08 −0.05 0.20 .25 321 0.11 −0.01 0.22 .08 321 0.07 −0.05 0.18 .24
PC aa C40:1
PC aa C40:2
PC aa C40:6
Neuropsychological tests n β 95% CI P value n β 95% CI P value n β 95% CI P value
Mini-mental state examination 388 0.01 −0.11 0.13 .85 388 −0.06 −0.16 0.03 .18 388 0.10 0.01 0.19 .04
Cognitive function domain
 Memory
 Delayed word recall 415 0.11 −0.02 0.24 .10 415 0.08 −0.03 0.18 .15 415 0.08 −0.02 0.18 .11
 Logical Memory Test part A 395 0.09 −0.04 0.23 .18 395 0.04 −0.07 0.15 .49 395 0.11 0.01 0.22 .03
 Logical Memory Test part B 385 0.05 −0.09 0.18 .51 385 0.05 −0.06 0.16 .37 385 0.09 −0.02 0.20 .10
 Incidental learning 378 0.12 −0.03 0.27 .11 378 0.12 0.01 0.23 .04 378 0.20 0.08 0.31 .001
 Language
 Animal naming 425 0.01 −0.12 0.13 .91 425 0.004 −0.10 0.11 .93 425 0.03 −0.06 0.13 .49
 Boston Naming Test 407 −0.03 −0.13 0.08 .59 407 −0.03 −0.12 0.05 .43 407 0.04 −0.04 0.12 .36
 Word fluency 407 −0.02 −0.15 0.11 .77 407 0.04 −0.07 0.14 .50 407 0.03 −0.07 0.13 .51
 Visuospatial
 Clock time perception 405 0.06 −0.09 0.20 .44 405 −0.03 −0.15 0.08 .55 405 0.06 −0.04 0.17 .24
 Attention
 Trail making test part A 379 −0.03 −0.14 0.09 .67 379 0.02 −0.08 0.11 .76 379 0.02 −0.07 0.12 .62
 Digit span backward 402 0.07 −0.06 0.20 .28 402 0.08 −0.03 0.18 .15 402 0.05 −0.05 0.15 .31
 Executive function
 Digit symbol substitution 389 0.001 −0.12 0.12 .98 389 −0.02 −0.11 0.08 .72 389 −0.01 −0.10 0.09 .88
 Trail making test part B 321 0.03 −0.12 0.18 .66 321 −0.04 −0.16 0.08 .49 321 0.06 −0.05 0.18 .28
PC ae C40:6
Lyso PC a C18:2
Propionyl-L-carnitine (C3)
Neuropsychological tests n β 95% CI P value n β 95% CI P value n β 95% CI P value
Mini-mental state examination 388 0.05 −0.04 0.15 .29 388 −0.10 −0.19 −0.01 .03 388 0.05 −0.04 0.13 .29
Cognitive function domain
 Memory
 Delayed word recall 415 0.06 −0.04 0.16 .27 415 0.01 −0.08 0.11 .77 414 0.03 −0.07 0.12 .59
 Logical Memory Test part A 395 0.12 0.01 0.23 .04 395 −0.09 −0.19 0.02 .10 394 0.04 −0.07 0.14 .50
 Logical Memory Test part B 385 0.11 0.003 0.23 .04 385 −0.03 −0.14 0.07 .55 384 0.04 −0.06 0.15 .44
 Incidental learning 378 0.18 0.07 0.30 .002 378 −0.02 −0.13 0.08 .66 378 −0.01 −0.12 0.10 .86
 Language
 Animal naming 425 0.04 −0.06 0.14 .42 425 −0.10 −0.19 −0.01 .04 424 −0.04 −0.13 0.05 .42
 Boston Naming Test 407 0.05 −0.03 0.14 .24 407 −0.03 −0.11 0.05 .49 406 0.03 −0.05 0.11 .42
 Word fluency 407 0.04 −0.07 0.14 .50 407 0.01 −0.08 0.11 .80 406 −0.003 −0.10 0.09 .95
 Visuospatial
 Clock time perception 405 0.05 −0.06 0.16 .40 405 −0.08 −0.18 0.03 .16 404 0.004 −0.10 0.11 .94
 Attention
 Trail making test part A 379 0.02 −0.08 0.12 .69 379 −0.01 −0.10 0.08 .82 379 −0.04 −0.13 0.05 .35
 Digit span backwards 402 0.13 0.03 0.24 .01 402 −0.003 −0.10 0.10 .95 401 −0.01 −0.11 0.09 .79
 Executive function
 Digit symbol substitution 389 0.02 −0.07 0.12 .63 389 0.03 −0.06 0.12 .51 389 −0.09 −0.18 −0.003 .04
 Trail making test part B 321 0.11 −0.01 0.24 .06 321 −0.01 −0.13 0.10 .84 321 −0.06 −0.18 0.05 .28

NOTE. Linear regression adjusted for age, race, sex, APOE, body mass index, diabetes mellitus, drinking status, educational level, HDL cholesterol, smoking status, sports index, systolic blood pressure, total cholesterol, triglycerides, use of antihypertensive medications, and prevalent coronary heart disease, heart failure, or stroke at baseline.

β per 1-SD change in the log(phospholipid).

β for phospholipid as a three-group ordinal variable (below LOD, ≥LOD− < Median, ≥Median).

Table 5.

Linear regression of plasma phospholipids PC aa C38:0 and PC ae C40:6 and cognitive scores of neuropsychological tests in older adults with normal cognitive status

Neuropsychological tests PC aa C38:0
PC ae C40:6
n β 95% CI P value n β 95% CI P value
Mini-mental state examination 130 0.06 −0.01 0.12 .095 130 0.04 −0.03 0.11 .26
Cognitive function domain
 Memory
 Delayed word recall 153 0.07 −0.05 0.18 .24 153 0.07 −0.05 0.20 .23
 Logical Memory Test part A 145 0.11 −0.03 0.26 .13 145 0.14 −0.01 0.29 .07
 Logical Memory Test part B 145 0.16 −0.001 0.32 .05 145 0.19 0.02 0.35 .03
 Incidental learning 149 0.41 0.24 0.57 <.0001 149 0.33 0.15 0.51 .0004
 Language
 Animal naming 153 0.14 0.01 0.28 .04 153 0.12 −0.02 0.27 .09
 Boston Naming Test 149 0.10 0.01 0.20 .03 149 0.09 −0.01 0.19 .07
 Word fluency 150 0.15 0.004 0.30 .04 150 0.13 −0.03 0.29 .10
 Visuospatial
 Clock time perception 149 0.04 −0.02 0.10 .19 149 0.04 −0.02 0.11 .18
 Attention
 Trail making test part A 144 0.08 −0.02 0.19 .12 144 0.02 −0.09 0.13 .70
 Digit span backward 149 0.19 0.05 0.34 .01 149 0.25 0.10 0.40 .002
 Executive function
 Digit symbol substitution 149 0.17 0.05 0.28 .004 149 0.14 0.02 0.26 .03
 Trail making test part B 140 0.25 0.09 0.41 .002 140 0.21 0.05 0.38 .01

Linear regression adjusted for multiple covariates including age, race, sex, and APOE, body mass index, diabetes mellitus, drinking status, educational level, HDL cholesterol, smoking status, sports index, systolic blood pressure, total cholesterol, triglycerides, use of antihypertensive medications, and prevalent coronary heart disease, heart failure, or stroke at baseline.

β per 1−SD change in the log(phospholipid).

4. Discussion

We performed targeted metabolomics of 188 metabolites (i.e., 40 acylcarnitines, 21 amino acids, 21 biogenic amines, 15 sphingolipids, 90 glycerophospholipids, and hexose) on plasma samples from 153 cognitively normal, 145 MCI, and 143 dementia participants in the multicenter ARIC-NCS study. We hypothesized that (a) lower concentrations of the 10 phospholipids identified by Mapstone et al. (PC aa C36:6, PC aa C38:0, PC aa C38:6, PC aa C40:1, PC aa C40:2, PC aa C40:6, and PC ae C40:6, lyso PC a C18:2, C3, and C16:1-OH) would be associated with a higher prevalence of MCI and dementia, and (b) that these phospholipids would help classify individuals as cognitively normal or impaired (MCI and/or dementia). Our multinomial logistic regression analyses adjusted for age, race, sex, APOE genotype, and additional cardiovascular risk factors indicated that, of the 10 phospholipids identified by Mapstone et al. only PC aa C40:2 and PC aa C36:6 were associated with prevalence of MCI and dementia, respectively. Being a hypothesis-driven analysis, we did not adjust for multiple testing for these phospholipids. Similarly, Klavins et al. [8] showed an association of lower concentrations of PC aa C36: 6 with MCI and dementia. In our analysis, the phospholipids identified by Mapstone et al. had weaker associations individually and in aggregate and a limited ability to differentiate those with dementia and/or MCI from the cognitively normal. Overall, other variables, including sociodemographic and cardiovascular risk factors, had better ability to detect MCI and dementia than the phospholipids of interest, as evaluated with C-statistics.

Higher concentrations of SM(OH) C22:1 were significantly associated with lower prevalence of dementia. Furthermore, higher concentrations of three additional sphingomyelins (SM C26:0, SM(OH) C22:2, SM(OH) C24:1) were similarly associated with lower prevalence of dementia, though these associations did not reach the Bonferroni-corrected level of statistical significance. Although dementia can have many etiologies, and AD is one of the many etiologies that may be responsible [18], our results are consistent with literature findings on the link between AD and sphingomyelins [19], [20].

Our study has several limitations. First, we used stored plasma samples. It is well known that pre-analytical variables in sample collection and processing may affect concentrations of phospholipids and metabolites in plasma [15], [8]. Kalvins et al. specifically tested stability of plasma phospholipids analyzed by the Biocrates p180 kits, which were the kits we used in this study and found if blood samples were processed immediately after collection and stored in −20°C, there were no statistically significant difference in the levels of plasma phospholipids between when these samples were analyzed on the same day of collection and when they were analyzed after 2 years of storage [8]. ARIC-NCS collected blood samples using standardized protocols, which instruct immediate blood processing after collection and storage of plasma samples at −80°C. In addition, these samples used in this study had been in storage for up to 3 years (in average <2 years) before they were analyzed. Furthermore, we used never-thawed plasma specimens to prevent the effects of freeze-thaw on specimen quality [15]. Therefore, the plasma specimens used in our study were of high quality and considered to be suitable for plasma phospholipids analysis. Second, we did not correct for multiple testing of the phospholipids in our primary analysis, because these phospholipids have been previously identified [7]. However, we did correct for multiple testing when we performed the exploratory, hypothesis-generating analysis of the rest of 151 metabolites. Third, we used the cross-sectional design, which did not allow the assessment of temporality. Finally, we had relatively small sample size to investigate the association of phospholipids with subtypes of dementia and/or MCI subtypes (AD vs. no-AD). Also, our clinical diagnoses of AD-type MCI and dementia lacked sufficient additional biomarker data to be certain about the AD etiology.

Although the same metabolomic method (i.e., the BioCrate p180 kits) was used in our study and by Mapstone et al. there were some key differences between the two studies. First, the Mapstone study was longitudinal in nature and was based on plasma samples collected from 124 cognitive normal individuals, 18 of who in 1–5 years (in average of 2.1 years) developed aMCI/AD. However, Mapstone et al. did report the ability of these 10 phospholipids in discriminating aMCI/AD from cognitive normal in a discovery set of 88 participants (35 aMCI/AD and 53 normal) and an independent validation of 41 participants (21 aMCI/AD and 20 normal). Our cross-sectional study of 441 participants had a bigger sample size than the Mapstone study (n = 88 and 41 for discovery and independent validation sets, respectively). Second, our study population was more heterogeneous, including 28% African-American, where the Mapstone study included only whites. In addition, our study population was less educated; Third, our study had expert adjudicated MCI and dementia diagnoses as well as etiologies, where the Mapstone study derived composite scores (standardized z-scores) for each participant on cognitive tests and categorized the participants into aMCI/AD or cognitive normal based on these composite scores. Fourth, the two studies used different statistic methods: our study used a multinomial logistic regression to assess the association of the individual metabolites with the prevalence of MCI and dementia; where the Mapstone study used LASSO analysis to build linear classifier models [7]. Differences in the statistical analyses could have a significant impact on results. Last but not least, pre-analytical process of blood samples is known to introduce variations in metabolites including phospholipids. As indicated in our study limitations, we used stored plasma samples, and Mapstone used prospectively collected samples. Although both studies collected plasma specimens using standardized protocols to ensure high quality and consistency within each study, differences in the sample collection protocols should be emphasized. Beside these key differences, plasma phospholipids may be affected by environmental factors, such as diet and exercise, and underlying mechanisms (e.g., inflammation and oxidative stress) of many diseases such as hypertension and cardiovascular diseases [21], [22], [23], [24]. To further add to the complexity of the associations between plasma phospholipids and cognition is the “dynamic” nature of lipidomics, because the populations of lipids within a cell membrane and within subcellular membranes changes constantly, and single assessment obtains only a snapshot picture of lipid profiles at the time of sample extraction, stopping the lipid-modeling process [25]. The dynamic nature of lipidomics really emphasizes the needs of validating biomarker studies with larger sample sizes, more heterogeneous study populations, and quality of MCI and dementia diagnosis, like we did in our study.

In conclusion, our study was unable to reproduce the identities of many phospholipids found to be significantly associated with prevalence of MCI and dementia in other studies. However, we found significant association of sphingomyelins with the prevalence of MCI and dementia. Our results showed that, in general, some phospholipids and metabolites were altered in MCI and dementia.

Research in context.

  • 1.

    Systematic review: Recent studies demonstrated that plasma phospholipids and metabolites might predict MCI and dementia [1], [2], [3], [4]. However, these studies were based on smaller sample sizes (i.e., 111–251 participants), included homogenous study populations (mostly European Caucasian), and had variable quality of mild cognitive impairment (MCI) and/or dementia characterization.

  • 2.

    Interpretation: By taking advantage of the rich neurocognitive data collected as part of the ARIC Neurocognitive Study (ARIC-NCS), we examined cross-sectionally whether concentrations of phospholipids/metabolites are associated with the prevalence of MCI or dementia in a subset of 441 white and African-American participants. Our findings suggested that although phospholipids and metabolites were altered in MCI and dementia, cross-sectional association was relatively weak and prediction of MCI and dementia did not improve on clinical variables.

  • 3.

    Future directions: Applying other metabolomics technologies on analysis of these specimens used in this study, we may discover additional metabolites that useful in the prediction of MCI and dementia.

Acknowledgments

We thank the staff and participants of the ARIC Study for their important contributions.

Sources of funding: The Atherosclerosis Risk in Community (ARIC) Study is carried out as a collaborative study supported by the National Heart, Lung, and Blood Institute contracts (HHSN268201100005C, HHSN268201100006C, HHSN268201100007C, HHSN268201100008C, HHSN268201100009C, HHSN2682011000010C, HHSN2682011000011C, and HHSN2682011000012C). Neurocognitive data are collected by the support of the National Heart, Lung, and Blood Institute U01 HL096812, HL096814, HL096899, HL096902, and HL096917 with previous brain MRI examinations funded by R01-HL70825.

Footnotes

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.dadm.2016.02.008.

Supplementary data

Supplementary Tables 1–5
mmc1.xlsx (70.7KB, xlsx)

References

  • 1.Klein J. Membrane breakdown in acute and chronic neurodegeneration: focus on choline-containing phospholipids. J Neural Transm. 2000;107:1027–1063. doi: 10.1007/s007020070051. [DOI] [PubMed] [Google Scholar]
  • 2.Pettegrew J.W., Panchalingam K., Hamilton R.L., McClure R.J. Brain membrane phospholipid alterations in Alzheimer's disease. Neurochem Res. 2001;26:771–782. doi: 10.1023/a:1011603916962. [DOI] [PubMed] [Google Scholar]
  • 3.Grimm M.O., Grösgen S., Riemenschneider M., Tanila H., Grimm H.S., Hartmann T. From brain to food: Analysis of phosphatidylcholins, lyso-phosphatidylcholins and phosphatidylcholin–plasmalogens derivates in Alzheimer's disease human post mortem brains and mice model via mass spectrometry. J Chromatogr A. 2011;1218:7713–7722. doi: 10.1016/j.chroma.2011.07.073. [DOI] [PubMed] [Google Scholar]
  • 4.Muldoon M.F., Ryan C.M., Sheu L., Yao J.K., Conklin S.M., Manuck S.B. Serum phospholipid docosahexaenonic acid is associated with cognitive functioning during middle adulthood. J Nutr. 2010;140:848–853. doi: 10.3945/jn.109.119578. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Beydoun M.A., Kaufman J.S., Satia J.A., Rosamond W., Folsom A.R. Plasma n−3 fatty acids and the risk of cognitive decline in older adults: the Atherosclerosis Risk in Communities Study. Am J Clin Nutr. 2007;85:1103–1111. doi: 10.1093/ajcn/85.4.1103. [DOI] [PubMed] [Google Scholar]
  • 6.Schaefer E.J., Bongard V., Beiser A.S., Lamon-Fava S., Robins S.J., Au R. Plasma phosphatidylcholine docosahexaenoic acid content and risk of dementia and Alzheimer disease: the Framingham Heart Study. Arch Neurol. 2006;63:1545–1550. doi: 10.1001/archneur.63.11.1545. [DOI] [PubMed] [Google Scholar]
  • 7.Mapstone M., Cheema A.K., Fiandaca M.S., Zhong X., Mhyre T.R., MacArthur L.H. Plasma phospholipids identify antecedent memory impairment in older adults. Nat Med. 2014;20:415–418. doi: 10.1038/nm.3466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Klavins K., Koal T., Dallmann G., Marksteiner J., Kemmler G., Humpel C. The ratio of phosphatidylcholines to lysophosphatidylcholines in plasma differentiates healthy controls from patients with Alzheimer's disease and mild cognitive impairment. Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring. 2015;1:295–302. doi: 10.1016/j.dadm.2015.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Olazarán J., Gil-de-Gómez L., Rodríguez-Martín A., Valentí-Soler M., Frades-Payo B., Marín-Muñoz J. A blood-based, 7-metabolite signature for the early diagnosis of Alzheimer's disease. J Alzheimers Dis. 2015;45:1157–1173. doi: 10.3233/JAD-142925. [DOI] [PubMed] [Google Scholar]
  • 10.Proitsi P., Kim M., Whiley L., Pritchard M., Leung R., Soininen H. Plasma lipidomics analysis finds long chain cholesteryl esters to be associated with Alzheimer's disease. Transl Psychiatry. 2015;5:e494–e502. doi: 10.1038/tp.2014.127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Investigators The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. The ARIC investigators. Am J Epidemiol. 1989;129:687–702. [PubMed] [Google Scholar]
  • 12.Schneider A.L., Sharrett A.R., Gottesman R.F., Coresh J., Coker L., Wruck L. Normative data for 8 neuropsychological tests in older blacks and whites from the Atherosclerosis Risk in Communities (ARIC) study. Alzheimer Dis Assoc Disord. 2015;29:32–44. doi: 10.1097/WAD.0000000000000042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Knopman D.S., Gottesman R.F., Sharrett A.R., Wruck L.M., Windham B.G., Coker L. Mild cognitive impairment and dementia prevalence: The Atherosclerosis Risk in Communities Neurocognitive Study. Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring. 2016;2:1–11. doi: 10.1016/j.dadm.2015.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Hainline A., Karon J., Lippel K. US Department of Health and Human Services; Bethesda, MD: 1982. Manual of Laboratory Operations: Lipid Research Clinics Program and Lipid and Lipoprotein Analysis; p. 628. [Google Scholar]
  • 15.Yin P., Peter A., Franken H., Zhao X., Neukamm S.S., Rosenbaum L. Preanalytical aspects and sample quality assessment in metabolomics studies of human blood. Clin Chem. 2013;59:833–845. doi: 10.1373/clinchem.2012.199257. [DOI] [PubMed] [Google Scholar]
  • 16.Hanley J.A., McNeil B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
  • 17.Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988;75:800–802. [Google Scholar]
  • 18.Albert M.S., DeKosky S.T., Dickson D., Dubois B., Feldman H.H., Fox N.C. The diagnosis of mild cognitive impairment due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement. 2011;7:270–279. doi: 10.1016/j.jalz.2011.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Mielke M.M., Bandaru V.V., Haughey N.J., Rabins P.V., Lyketsos C.G., Carlson M.C. Serum sphingomyelins and ceramides are early predictors of memory impairment. Neurobiol Aging. 2010;31:17–24. doi: 10.1016/j.neurobiolaging.2008.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Mielke M.M., Haughey N.J., Bandaru V.V.R., Weinberg D.D., Darby E., Zaidi N. Plasma sphingomyelins are associated with cognitive progression in Alzheimer's disease. J Alzheimers Dis. 2011;27:259. doi: 10.3233/JAD-2011-110405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Haskell W.L. The influence of exercise training on plasma lipids and lipoproteins in health and disease. Acta Med Scand Suppl. 1986;220:25–37. doi: 10.1111/j.0954-6820.1986.tb08929.x. [DOI] [PubMed] [Google Scholar]
  • 22.Lands W.E., Libelt B., Morris A., Kramer N.C., Prewitt T.E., Bowen P. Maintenance of lower proportions of (n− 6) eicosanoid precursors in phospholipids of human plasma in response to added dietary (n− 3) fatty acids. Biochim Biophys Acta. 1992;1180:147–162. doi: 10.1016/0925-4439(92)90063-s. [DOI] [PubMed] [Google Scholar]
  • 23.Grimsgaard S., Bønaa K.H., Jacobsen B.K., Bjerve K.S. Plasma saturated and linoleic fatty acids are independently associated with blood pressure. Hypertension. 1999;34:478–483. doi: 10.1161/01.hyp.34.3.478. [DOI] [PubMed] [Google Scholar]
  • 24.Lemaitre R.N., King I.B., Mozaffarian D., Sotoodehnia N., Rea T.D., Kuller L.H. Plasma phospholipid trans fatty acids, fatal ischemic heart disease, and sudden cardiac death in older adults the cardiovascular health study. Circulation. 2006;114:209–215. doi: 10.1161/CIRCULATIONAHA.106.620336. [DOI] [PubMed] [Google Scholar]
  • 25.Brown H.A., Murphy R.C. Working towards an exegesis for lipids in biology. Nat Chem Biol. 2009;5:602–606. doi: 10.1038/nchembio0909-602. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Tables 1–5
mmc1.xlsx (70.7KB, xlsx)

Articles from Alzheimer's & Dementia : Diagnosis, Assessment & Disease Monitoring are provided here courtesy of Wiley

RESOURCES