MANUSCRIPT IN PRESS: DEMENTIA & GERIATRIC COGNITIVE DISORDERS

Abstract

Background

Prior work on the link between blood-based biomarkers and cognitive status has largely been based on dichotomous classifications rather than detailed neuropsychological functioning. The current project was designed to create serum-based biomarker algorithms that predict neuropsychological test performance.

Methods

A battery of neuropsychological measures was administered. Random forest analyses were utilized to create neuropsychological test-specific biomarker risk scores in a training set that were entered into linear regression models predicting the respective test scores in the test set. Serum multiplex biomarker data were analyzed on 108 proteins from 395 participants (197 AD cases and 198 controls) from the Texas Alzheimer’s Research and Care Consortium.

Results

The biomarker risk scores were significant predictors (p<0.05) of scores on all neuropsychological tests. With the exception of premorbid intellectual status (6.6%), the biomarker risk scores alone accounted for a minimum of 12.9% of the variance in neuropsychological scores. Biomarker algorithms (biomarker risk scores + demographics) accounted for substantially more variance in scores. Review of the variable importance plots indicated differential patterns of biomarker significance for each test, suggesting the possibility of domain-specific biomarker algorithms.

Conclusions

Our findings provide proof-of-concept for a novel area of scientific discovery, which we term “molecular neuropsychology.”

Background

The long-standing search for accurate biomarkers from many conditions/diseases impacting neuropsychological functioning include, but are not limited to, Alzheimer’s disease (AD)[1–4], traumatic brain injury (TBI) [5, 6], schizophrenia [7, 8], alcohol use/abuse [9], and mood disorders [9–11]. For example, we recently created a serum-based biomarker algorithm that yielded excellent diagnostic accuracy in separating AD cases from controls [1, 12, 13]. Significant advancements have been made though analyses of blood, cerebrospinal fluid, and advanced neuroimaging modalities, and it is likely that combining assessment modalities (e.g. biomarkers + clinical data + demographics) will yield better results than any single modality [1, 12, 14–16].

While animal model work has begun to examine proteomic and genomic methods for discovering potential pathways and biomarkers of specific cognitive abilities [17], the majority of human biomarker research to date has been based on dichotomous group classifications (case vs. control) rather than linear constructs upon which clinical decisions and/or diagnoses of cognitive dysfunction are based (i.e. neuropsychological testing). While others are examining the link between biomarkers of disease states (e.g. AD) and cognitive functioning [18], biomarkers associated with neuropsychological test performance as continuous variables could provide novel opportunities to study these conditions/diseases[4]. For example, in our prior work we did not show a significant difference in serum BDNF levels between AD cases and controls; however, serum BDNF levels were specifically associated with memory performance among AD cases[4]. Blood-based biomarkers are preferable as they are more cost and time efficient and more conveniently acquired than CSF or neuroimaging [19]. An additional advantage of proteomic approaches to biomarker identification is the potential to discover alterations at the protein level that may be closely related to the pathophysiological process(es) underlying complex conditions and disease states [9]. The purpose of the present study was to take a first step towards creating serum-based biomarker algorithms of neuropsychological functioning. This proof of concept project provides a platform for a novel field of scientific discovery, which we term “molecular neuropsychology.”

Even though there are currently no available blood-based biomarkers of neuropsychological functioning, there is a growing literature linking specific blood biomarkers to neuropsychological performance. In the instance of AD, we analyzed data from a sample of 399 participants (198 AD, 201 controls) enrolled in the Texas Alzheimer’s Research & Care Consortium (TARCC) and found that serum brain-derived neurotrophic factor (BDNF) levels were significantly associated with poorer immediate and delayed visual and verbal memory scores[4] among AD cases but not controls. We failed to find a significant link between C-reactive protein (CRP) levels and cognition (Mini-Mental State Examination, MMSE[20]) scores among a sample of 192 AD cases and 174 controls, though increased CRP levels were associated with disease severity [21]. On the other hand, Noble and colleagues [21] analyzed data on 1,331 participants of the WHICAP study and found CRP levels (highest tertile versus lowest tertile) to be specifically associated with impairment in memory and visuospatial abilities but not language or executive functioning. Additionally, it was noted that ApoEμ4 carriers were most likely to demonstrate impairment in memory. Wilson et al [22] examined a sample of controls, MCI and dementia cases (AD and non-AD dementias) with CDR global scores ranging from 0 to 3. These authors found that Repeatable Battery for Neuropsychological Status (RBANS)[23] Language, Immediate and Delayed Memory Indices were significantly related to plasma Anti-RAGE immunoglobulins (IgGs) and Anti-Aβ IgGs concentrations. Interestingly, Igμ, a heavy chain of immunoglobulin M (IgM) was part of the protein profile recently identified in serum among schizophrenia patients [8] and immunoglobulins were also part of our AD diagnostic algorithm (Appendix 1 of O’Bryant et al[1]). While neuropsychological testing was not conducted as part of the schizophrenia project, this work suggests a role of immunoglobulins in neuropsychological functioning across multiple disease states. When examining the outcome of children suffering from traumatic brain injury with and without histories of abuse, Beers et al [24] found serum concentrations of neuron-specific enolase (NSE), S100B, and myelin-basic protein (MBP) were related to six-month functional and cognitive outcomes. In a sample of 998 non-demented community-dwelling adults of the Longitudinal Aging Study Amsterdam (LASA), Dik and colleagues [25] found that adrenocorticotropic hormone (ACT) concentration was significantly associated with delayed verbal recall and albumin with MMSE scores. When the sample was restricted to those with MMSE scores greater than 21 (suggesting mild to moderate dementia), ACT was also associated with processing speed and delayed recall. Next the authors examined only those suspected of possible mild cognitive dysfunction (i.e. MMSE scores 21–26) and found that IL-6 and CRP were significantly associated with cognitive decline over a three-year period. Despite the growing research linking specific blood-based biomarkers and neuropsychological test performance (by cognitive domain in many cases) across various conditions and disease states, no prior work has attempted to create biomarker algorithms of test-specific neuropsychological functioning.

The current study sought to create neuropsychological test-specific biomarker algorithms in a cohort of elders with and without cognitive impairment. The ability to create such algorithms would have broad reaching applications. We hypothesized that our test-specific algorithms would account for significant percentages of the variance in neuropsychological performance in the test sample.

Methods

Participants

Participants included 395 individuals (197 AD subjects, 198 controls) enrolled in the Texas Alzheimer’s Research & Care Consortium (TARCC). The methodology of the TARCC project has been described in detail elsewhere[1, 26]. Briefly, each participant undergoes a standardized annual examination at the respective sites, which includes a medical evaluation, neuropsychological testing, interview, and blood draw for storage of samples in the TARCC biobank. Diagnosis of AD was based on NINCDS-ADRDA criteria[27] utilizing consensus review. Controls were subjects without cognitive complaints or medical conditions that could impair cognition and who performed within normal limits on psychometric assessment. Institutional Review Board approval was obtained at each TARCC site and written informed consent was obtained for all participants.

Assays

Non-fasting blood samples were collected into 10mL tiger-top serum-separating tubes. Samples were allowed to clot at room temperature for 30 minutes in a vertical position. Within one hour of collection, samples were centrifuged for 10 minutes at 2500 × g and aliquoted into 1mL cryovial tubes and stored in −80°C freezers. Batched specimens from either baseline or year-one follow-up exams were sent frozen to Rules Based Medicine (RBM, www.rulesbasedmedicine.com, Austin, TX) where they were thawed for assay without additional freeze-thaw cycles using the RBM multiplexed immunoassay human Multi-Analyte Profile (humanMAP). Individual proteins were quantified with immunoassays on colored microspheres. Information regarding the mean, standard deviation, least detectable dose (LDD), and inter-run coefficient of variation, are listed in Table 1.

Table 1.

Mean, standard deviation, and least detectable dose for top blood-based markers in algorithms

	Mean	Std	LDD
VCAM-1	799.79	213.14	1.5ng/mL
B2M	2.39	1.04	0.013ug/mL
IL8	22.95	8.87	3.5pg/mL
vWF	38.26	21.32	0.40ug/mL
Eotaxin3	104.13	333.29	70.0pg/mL
TenascinC	957.12	639.29	3.0ng/mL
PPY	192.10	191.66	5.0pg/mL
FAS	8.88	4.11	0.27ng/mL
A2M	1.33	1.06	0.061mg/mL
TPO	6.11	1.83	3.2ng/mL
S100b	0.43	0.32	0.30ng/mL
CKMB	0.36	0.34	0.42ng/mL
SCF	526.85	209.58	56.0pg/mL
EGF-R	4.93	1.14	0.042ng/mL
TNFb	3.50	3.61	46.0pg/mL
AgRP	73.46	130.41	.165ng/mL
G-CSF	10.06	4.84	3.4pg/mL
CRP	3.45	4.52	0.0015ug/mL

NOTE: LDD = least detectable dose

Neuropsychological Testing

The TARCC neuropsychology core battery consists of commonly utilized instruments that tap a variety of cognitive domains. Specific tests include Digit Span (WAIS-R, WAIS-III, WMS-R)[28], Trail Making Test[29], Logical Memory and Visual Reproduction (WMS-R and WMS-III)[28], Boston Naming Test (30- and 60-item versions)[29], verbal fluency (FAS)[30], Clock Drawing Test[30], the American National Adult Reading Test (AMNART)[30], MMSE[20], and the Clinical Dementia Rating scale (CDR)[31]. In order to equate scores from digit span and story memory scales, raw scores for all neuropsychological tests were converted to scaled scores based on previously published normative data [32–34]. For the Boston Naming Test, we recently published an independent study demonstrating the psychometric properties of an estimated 60-item BNT score that can be calculated from 30-item versions[35]; this estimated 60-item score was calculated for all 30-item administrations.

Statistical Analyses

Analyses were performed using R (V 2.10) statistical software[36]. The log transformed and then standardized data on each analyte from our AD diagnostic algorithm publication was utilized in the current analyses[1]. Random forest (RF) analysis was developed by Breiman as an ensemble learning method that utilizes a classification tree as the base classifier[37, 38]. The RF model has four steps: 1. generate many random subsets (bootstrap samples) from the original data; 2. build a decision tree from each random subset; 3. make a prediction from each decision tree model; 4. combine the predictions from each individual tree model to get the final prediction. The RF model can be used to predict both binary/categorical outcomes and continuous outcomes. For predicting binary or categorical outcomes, the random forest model will build many classification trees, and then combine the predictions from individual trees by a majority vote approach. While for predicting continuous outcomes, the random forest model will build regression trees [2] instead of classification trees, and then use model averaging techniques to combine the predictions from individual regression trees. This method has been shown to perform well in many classification and prediction scenarios[39, 40], including algorithmic approaches CSF[41], EEG[42] and fMRI [43, 44] findings. The random forest prediction model was performed using R package randomForest (V 4.5)[37], with all software default settings. TARCC participants were randomized into a training set (n=197, AD n=98, control n=99) or a testing set (n=198; AD n=99, control n=99) by random number generator. The full list of 108 serum-based analytes utilized in the algorithm can be found in Appendix 1. A random forest (RF) prediction model biomarker risk score was generated for each specific neuropsychological test within the training set, which was then entered as predictor variables in linear regression models in the test set, with the neuropsychological scale scores as the outcome variables; age, gender, and education were entered as covariates. The percentage of participants having a diagnosis of diabetes, hyperlipidemia, hypertension, or being obese is presented in Table 2. With the exception of obesity (AD = 13%, controls = 21%), there was no significant difference in presence of these conditions between groups. Obesity status was not significantly related to neuropsychological test scores and, therefore, none of these medical conditions were included in our analyses. Significance was set at p<0.05.

Table 2.

Demographic Characteristics of Sample

	Total Sample (mean, sd)
Gender (male)	33%
Age	73.9 (9.3)
Education	14.8 (3.2)
MMSE	24.4 (6.8)
CDR SB	3.8 (5.0)
APOE4 carrier (yes/no)	163/218 (19 unknown)
Hispanic Ethnicity	9%
Race
White	94%
Non-White	6%
Diabetes (yes)	11%
Hypertension (yes)	60%
Hyperlipidemia (yes)	57%
Obesity (yes)	17%

Note: MMSE = Mini-mental Sate Examination; CDR SB = Clinical Dementia Rating scale – Sum of Boxes score; apoE = apolipoprotein € 4 allele

Results and Discussion

Demographic characteristics of the study population are shown in Table 2. The distributions of neuropsychological test scores by AD versus control status can be found in Figure 1. The relation between the most relevant biomarkers and neuropsychological test scores can be found in the heatmap of Figure 2. Online supplemental Figure 3 provides the distributions of biomarkers by AD versus control status.

Figure 1.

Distribution of Neuropsychological Test Scores by AD versus normal control

Figure 2.

Heatmap of correlation coefficients between individual biomarkers and neuropsychological test scores

The biomarker risk scores created from the training set were significant predictors of all neuropsychological scores in the test set. However, the biomarker risk scores for global cognition (MMSE, p<0.001) and disease severity (CDR Global Score p<0.001, CDR Sum of Boxes score p<0.001) were among the most powerful. Across the domain of memory, the biomarker risk scores were strong predictors of test performance in both visual (Visual Reproduction I p<0.001, Visual Reproduction II p<0.001) and verbal domains (Logical Memory I p<0.001, Logical Memory II p<0.001) (see Table 3 for all results).

Table 3.

Regression results predicting neuropsychological scores in the test set from the biomarker risk scores and biomarker algorithm created in the training set

	B	SE	p-value	% Variance – Biomarker Risk Score	% Variance – Full Biomarker Algorithm
MMSE	1.29	0.15	<0.001	28.7	37.6
CDR SB	1.53	0.15	<0.001	33.5	42.3
CDR GS	1.51	0.15	<0.001	34.3	41.6
CLOCK	1.82	0.24	<0.001	24.6	32.0
COWAT	1.49	0.22	<0.001	20.6	36.7
BNT	1.38	0.22	<0.001	17.5	26.5
AMNART	0.95	0.27	<0.001	6.6	37.0
Trails A	1.80	0.26	<0.001	21.7	30.1
Trails B	1.67	0.27	<0.001	21.0	27.7
Digit Span	1.24	0.23	<0.001	12.9	22.4
WMS LM Immediate recall	1.86	0.17	<0.001	43.9	49.4
WMS LM Delayed Recall	1.83	0.15	<0.001	47.3	51.2
WMS VR Immediate Recall	1.59	0.18	<0.001	30.9	35.3
WMS VR Delayed Recall	1.99	0.18	<0.001	41.2	44.7

Note: B = unstandardized beta; SE = standard error; % variance – biomarker risk score = % variance in neuropsychological test scores accounted for by biomarker risk score alone; % variance full biomarker algorithm = variance accounted for in neuropsychological test scores by biomarker risk score + demographics (age, gender, and education); MMSE = Mini-Mental State Examination; CDR SB = clinical dementia rating scale sum of boxes score; CDR GS = clinical dementia rating scale global score; CLOCK = clock drawing task; COWAT = Controlled Oral Word Association Test scale score; BNT = Boston Naming Test scale score; AMNART = American North American Reading Test scale score; Trails A = Trail Making Test part A scale score; Trails B = Trail Making Test part B scale score; Digit Span scale score; WMS = Wechsler Memory Scale, LM = logical memory, VR = visual reproduction, all scale scores.

The amount of variance in test scores accounted for by the biomarker risk scores alone ranged from 6.6% to 47.3%. In fact, the biomarker risk score accounted for 43.9% of the variance in immediate verbal memory (WMS LM I) and 47.3% in delayed verbal memory (WMS LM II), 30.9% of the variance in immediate visual memory (WMS VR I) and 41.2% of the variance in delayed visual memory (WMS VR II). The amount of variance accounted for by each test-specific biomarker risk score, independent of age, gender and education is presented in Table 3. Prior work has demonstrated that age, gender, and education influence neuropsychological test performance [29, 30] and it is standard practice to use these factors in creating normative references. Therefore, we also examined the variance accounted for in neuropsychological test scores by test-specific biomarker algorithms (biomarker risk scores + age, gender and education). The biomarker algorithms accounted for large portions of variance in neuropsychological test scores including 49.4%–51.2% of the variance in verbal memory, 33.5%–44.7% of the variance in visual memory, 26.5%–36.7% of the variance in language, 27.2%–32.0% of the variance in executive functioning, and 30.1% of the variance in processing speed. We also created a biomarker algorithm that accounted for 41.6%–42.6% of the variance in disease severity (see Table 3). The amount of additional variance accounted for by inclusion of demographic factors varied by test and was smallest for memory measures.

Lastly, we reviewed the variable importance plots. Random forests generate variable importance plots based on the number of times each variable is selected by all individual trees in the ensemble (naive variable importance) [45]. The Gini importance method, utilized in the current study, incorporates a weighted mean of the improvement in the splitting criteria of the individual trees produced by each variable [45]. Review of the variable importance plots showed that the relative ranking of markers in the algorithm varied by every test-specific biomarker risk score in the analyses. To illustrate this, variable importance plots reflecting the top 10 markers contained in the algorithms of tests from four different domains (executive functioning, memory, intelligence, and attention) are presented in Figures 3 through 6. It is worth noting that the Gini index relies on the assumptions of independence, and does not account for the dependency among the features. More advanced statistical methods may be need to more accurately access the modeling fit.

Figure 3.

Variable importance plot for the domain of executive functioning

Figure 6.

Variable importance plot for the domain of attention

Conclusion

We took a novel approach to the search for biomarkers of cognitive functioning by shifting the outcome variable from dichotomous categorization of group status (case versus control) to a continuous construct of neuropsychological test scores. Here we demonstrate that it is possible to create serum-based biomarker algorithms of specific neuropsychological function that account for considerable portions of the variance in test scores and that the biomarker profiles vary according to cognitive domain. This proof-of-concept work supports expanding the search for biomarker mediators of cognitive functioning, which we entitle Molecular Neuropsychology.

This work expands on prior work looking at how individual biomarkers are related to specific neuropsychological tests, which may have broader utility for understanding and predicting cognitive dysfunction. For example, it is possible that biomarker profiles will aid in the identification of those at greatest risk cognitive decline. Prior work has demonstrated that baseline neuropsychological test scores predict change in status over time as well as progression to AD. For example, Musicco and colleagues [46] recently analyzed data on 154 newly diagnosed AD cases and found that more severe memory and executive functioning difficulties at baseline predicted more rapid progression over a two-year follow-up period. Examining data from the Vienna Transdanube Aging Study (VITA), Jungwirth et al [47] analyzed 5-year longitudinal data to determine what baseline variables best predicted incident AD in those without cognitive impairment. These authors found that a combination of baseline CERAD Word List Delayed Recall, Trail Making Test part A, presence of the ApoE μ4 allele gene, and memory complaints significantly predicted incident AD, with an area under the receiver operating characteristic curve (AUC) of 0.91 and a model R² = 0.43. It is possible that combining biomarkers with select cognitive and clinical variables will significantly improve the prediction capacity of such models and our group is currently working on this possibility. Another advantage of this approach is the possibility of studying biological systems that may be of etiological importance. Lastly, a significant amount of research has gone into imaging neurological mechanisms of cognitive function/dysfunction. The current work may provide the ability to combine biochemical pathway analysis with functional neuroimaging methods for a better understanding of the biology of neuropsychological dysfunction.

The current results also support the notion that our algorithms may be test-specific based on the variable importance plots. In fact, the order of the markers varied across all neuropsychological tests (Clock, WMS LM I, AMNART, and Digit Span presented in Figures 1–4). While it has been suggested that the variable importance measures from random forest analysis can be biased [45], the current findings provide ample proof-of-concept for delving further into the investigation of neuropsychological domain-specific blood-based biomarker algorithms. While we utilized a broad set of proteins, there are other proteins that have been found important that were not included in our multi-plex assays [22]. It is certainly possible that better algorithms can be created by utilization of even larger protein panels that could then be refined to briefer versions.

Figure 4.

Variable importance plot for the domain of memory

There are limitations to the current study. While we utilized a broad range of serum-based proteins, it is likely that other markers not included in our assays would contribute significantly to building such blood-based biomarker algorithms of neuropsychological functioning. Future work should utilize a larger discovery set of biomarkers that can be narrowed as necessary. A second limitation to the current study is the brevity of the neuropsychological test battery. The TARCC battery consists of commonly used neuropsychological instruments; however, it was designed to be brief and does not fully assess several domains of cognition (e.g. executive functioning, visuospatial skills). In order to thoroughly research the potential of creating serum-based biomarker algorithms of neuropsychological tests and/or domains, a more comprehensive battery of tests should be implemented. A second limitation to the TARCC battery is the lack of consistency across sites with regards to instruments (e.g. WMS-R versus WMS-III). We utilized scale scores in order to equate across test versions; however, this remains a limitation to the protocol. Our study is also limited by inclusion of only AD cases and normal controls. When examining only individuals who have been screened into research projects based on performing “normal” or abnormal on cognitive measures creates an inherent bias that reduces the range of performance in neuropsychological test scores, as is the case with the TARCC cohort. Inclusion of AD cases in the current study enables us to capitalize on a full spectrum of cognitive performance. However, AD status was utilized in case selection thereby causing some unavoidable circularity. On the other hand, if one enters disease severity (or disease status) into the models, the results do not hold, which is likely due to the fact that disease severity and degree of neurocognitive deficits are highly confounded. If we analyzed the group separately, the current findings do not hold due to this restricted range of scores. The advantage of this approach is the ability to have a broad range of cognitive function/dysfunction; however, additional work is needed to represent a full spectrum of neuropsychological status. For example, there is evidence to suggest that biomarkers will also be related to exceptional cognitive functioning as demonstrated by Lopez et al [48] who found that uromodulin and Compmlement C3 were expressed at higher levels among highly intelligent elders when compared to those with lower intelligence.

The current study demonstrates that (1) blood-based biomarkers can be combined to create algorithms related to neuropsychological functioning, and (2) biomarker profiles will vary according to the cognitive domain being examined. This proof-of-concept work highlights the importance of investigating blood-based biomarker profiles of neurosychological functioning, which may have clinical utility across a broad range of conditions/populations.

Figure 5.

Variable importance plot for the domain of intelligence