Improving Brain Age Prediction Models: Incorporation of Amyloid Status in Alzheimer’s Disease

Abstract

Brain age prediction is a machine learning method that estimates an individual’s chronological age from their neuroimaging scans. Brain age indicates whether an individual’s brain appears “older” than age-matched healthy peers, suggesting that they may have experienced a higher cumulative exposure to brain insults or were more impacted by those pathological insults. However, contemporary brain age models include older participants with amyloid pathology in their training sets and thus may be confounded when studying Alzheimer’s disease (AD). We showed that amyloid status is a critical feature for brain age prediction models. We trained a model on T1-weighted MRI images participants without amyloid pathology. MRI data were processed to estimate gray matter density voxel-wise, which were then used to predict chronological age. Our model performed accurately comparable to previous models. Notably, we demonstrated more significant differences between AD diagnostic groups than other models. Additionally, our model was able to delineate significant differences in brain age relative to chronological age between cognitively normal individuals with and without amyloid. Incorporation of amyloid status in brain age prediction models ultimately improves the utility of brain age as a biomarker for AD.

1. Introduction

Neuroimaging-based brain age prediction may serve as a promising, individualized biomarker of brain health (Cole and Franke 2017) to understand the highly heterogeneous biological changes that occur in aging. Machine learning brain age prediction models learn the association between age and neuroimaging data in healthy individuals, where brain age is approximately equal to chronological age in healthy individuals. Once trained, the brain age model may be utilized in independent samples as a marker of brain health. If the resulting brain age is lower than chronological age, that individual may have a “younger” brain than expected and may be more resistant to or have accumulated less pathology. Alternatively, if predicted brain age is greater than chronological age, that individual may have an “older” brain than expected, and may have a genetic predisposition, or have experienced a higher cumulative exposure to brain insults. These individuals may have been more impacted by pathological insults due to less effective homeostatic mechanisms compared to their age-matched peers. Brain age prediction models have demonstrated the association of increased brain age with cognitive impairment, Alzheimer’s disease (AD), traumatic brain injury, Down’s syndrome, HIV, and more (Behehsti 2018, Liem 2017, Cole 2015, Cole 2017, Gaser and Franke 2013).

Prior brain age models have been limited due to the inclusion of amyloid positive older participants in the training sets. It has been demonstrated that amyloid (Aβ) deposition, which is a hallmark of AD, may occur decades prior to the clinical onset of AD (Sperling 2011, Jack 2013). However, the neurotoxic effects of amyloid may be exerted on the brain for years before manifestation of overt cognitive impairment (Aizenstein 2008). In this period of time before cognitive symptoms appear, Aβ(−) and Aβ(+) individuals demonstrate subtle structural or functional differences, such as grey matter (Mattsson 2014) and white matter atrophy (Vipin 2019), cerebral hypometabolism (Bozoki 2014), disruptions in grey matter networks (Ten Kate 2018), default mode network, and the central executive network (Lim 2014). It is possible that brain age models not accounting for amyloid status in the training set (Aβ insensitive models) may not detect these very subtle differences and distinguish between individuals in these stages. Therefore, we trained a brain age model with only cognitively normal, Aβ(−) individuals to potentially improve the utility of brain age as a biomarker in the context of aging and AD. We then applied multivariable regression modeling to examine for differences between AD cognitive diagnostic stages, as well as amyloid status among cognitively healthy participants. In addition, we compared our results against brain ages obtained from a well-known but amyloid insensitive brain age model (Cole 2015, Cole 2017). We hypothesized that the association between brain age and chronological age would be moderated by group – specifically we hypothesized that the Aß(+) group would have a greater positive association between brain age and chronological as compared to the Aß(−) group indicating a more rapid aging process (cross-sectionally). We further hypothesized that more severe diagnostic groups (CN < EMCI < LMCI < AD) would have more rapid associations between brain and chronological age.

2. Methods

2.1. Data cohorts

This study included a total of 1256 structural magnetic resonance imaging (MRI) scans from a combination of publicly available databases (full details in supplement). All scans were acquired using standard T1-weighted sequences. See supplement for more detailed cohort information.

2.2. Training set

The training set consisted of 757 images from healthy, Aβ(−) individuals from the ADNI (n=92, mean age 73.9, range 60–85, 3T), IXI (n=264, mean age 34.9, range 20–49, 1.5 and 3T), and OASIS-3 (n=401, mean age 66.0, range 42–85, 3T) datasets (Supplemental Table 1). Inclusion criteria were the age range of 20–85, normal cognitive function, and sustained global amyloid-PET negativity over at least 3 years (only for participants of age 50+ since detectable amyloid deposition is almost non-existent in individuals without genetic mutations <50 years old). The earliest corresponding T1 with negative amyloid status was selected for use in the training set. Participants were excluded for cognitive impairment, memory complaint, dementia, history of psychosis or neurologic disorders, and contraindications to MRI and PET imaging.

2.3. Test sets

The test sets (Supplemental Tables 2 and 3) consisted of 491 3T T1-images from six groups: 1) cognitively normal (CN) and Aβ(−) individuals from the ADNI dataset (CN-Aβ(−), n=51); 2) CN, Aβ(−) individuals from the Pittsburgh community dataset (CN-Aβ(−) PITT, n=32); 3) ADNI CN, Aβ(+) individuals (CN-Aβ(+), n=51); 4) ADNI early mild cognitive impairment (MCI) individuals (EMCI, n=195); 5) ADNI late MCI (LMCI, n=88); and 6) ADNI AD individuals (AD, n=74). The CN-Aβ(−) group was used as an independent validation set for the model. All participants were between the age of 60–85. All CN-Aβ(−) individuals sustained Aβ-PET negativity over 3 years, while CN-Aβ(+) individuals demonstrated Aβ-PET positivity. The CN-Aβ(−) PITT cohort was included as an additional community-based comparison against the ADNI CN cohorts. All CN cohorts were matched by age (mean age matched). EMCI, LMCI, and AD groups were all Aβ(+) and are described in ADNI protocols. No test set participants were used in training of the model.

2.4. Image pre-processing

Using the Statistical Parametric Mapping (SPM12) software package, structural images were segmented into tissue classes (gray, white, cerebrospinal fluid, skull, soft-tissue, and air). We utilized the nonlinear DARTEL (fast diffeomorphic registration) algorithm to register images to the Montreal Neurological Institute (MNI) space then generated a template per cohort, and then smoothed with a 4 mm smoothing kernel.

2.5. Machine learning model creation and validation

The Pattern Recognition for Neuroimaging Toolbox (PRoNTo) (Schrouff 2013) was used to create the machine learning model. Whole brain, voxel-wise grey matter densities were mean-centered and then were used to compute a similarity matrix kernel – in particular we used the simple dot product (this is an NxN matrix that estimates the distance or similarity between any two participants). This matrix was used in a Gaussian Processes Regression model with the similarity matrix as the independent variable and chronologic age as the dependent variable with cohort (i.e., ADNI, IXI, or OASIS-3) as a covariate. Accuracy of the machine learning model was assessed by running a 10-fold cross-validation on the training set. We permuted chronological age (500 permutations) to assess the significance of the model prediction. Since each fold may result in slightly different models, the final overall model was an average of the 10-fold cross-validation. This average model was then tested on a separate independent test set of CN-Aβ(−), which served as a hold-out test set. We also assessed the validity of brain age prediction by comparing brain age between CN-Aβ(−), CN-Aβ(−) PITT, CN-Aβ(+), EMCI, LMCI, and AD participants. The CN-Aβ(−), CN-Aβ(−) PITT, CN-Aβ(+), EMCI, LMCI, and AD participants were not part of the training in any way.

2.6. Comparison against amyloid insensitive brain age model

We also computed brain age using a model that has been previously described, validated, and widely implemented in previous literature which does not account for amyloid status in its training (Cole 2015, Cole 2017). Code for the model was downloaded from https://github.com/james-cole/brainageR. This was used to estimate predicted brain age based on each participant’s gray matter and white matter data. We used this brain age measure as an amyloid-insensitive brain age compared to our model that accounted for amyloid.

2.7. Statistical Analysis

All statistical analyses were conducted in JMP Pro 14.1.0 (SAS Institute Inc., 2018). Multivariable regression modeling was used to determine the effects of chronological age (CA), group, and CA-group interactions on brain age. This was performed for the AD cognitive diagnostic stages (CN, EMCI, LMCI, and AD), as well as CN subgroups of differing amyloid status (CN-Aβ(−), CN-Aβ(−) PITT, CN-Aβ(+)). Non-significant interactions were removed from the final model. Values of r, R² and mean absolute error (MAE) were evaluated for the goodness of fit of the model. The same analyses were performed on the results of the amyloid insensitive model.

Difference between brain age and chronological age has been previously utilized to identify differences, however recently it has been shown that the strength of the correlation between brain age and chronological age does not guarantee that the difference will be estimated accurately (Smith 2019). Further, the difference between brain age and chronological age is not orthogonal to chronological age – this means that factors correlated with age may be falsely associated with difference between brain and chronological age (Smith 2019). By modeling brain age statistically with chronological age as a predictor, we circumvent the need to compute this brain age and chronological age difference. This also allows for quadratic associations with chronological age to be modeled (we did not do this here).

2.8. Cross-Validated Prediction of Groups using Logistic Classifier

To help understand the predictive potential of these features, we trained a simple logistic classifier for predicting the groups (CN-Aß(−), CN-Aß(+), EMCI, LMCI, and AD) using the following: chronological age alone; brain age alone; and chronological age and brain age. Since there is a high number of individuals in the EMCI group, we decided to include 30 participants from each group to help balance our logistic classifier (as to not overfit on EMCI). We conducted following analysis 500 times: (1) choose a random set of 30 participants from each group; (2) train the logistic classifier on this training set of 150; and (3) output predictions on other individuals not in training set. We then computed the average prediction for each participant across the 500 repetitions. We then evaluated the model’s performance based on area under the curve (AUC), accuracy, sensitivity, and specificity. Since the number of participants in the EMCI group is large, we needed a baseline model to help understand what performance we need to improve upon. We used the ZeroR model, which predicts groups by choosing the most common category (i.e., each participant is identified as EMCI) – this is a baseline model that evaluates how good of an accuracy, sensitivity, and specificity we need to improve upon chance prediction. We describe this model in Supplementary Figure 4. We conducted the same predictions using the Cole model as well.

3. Results

Training set demographics are shown in supplemental table 1, test set demographics between cognitive groups (CN, EMCI, LMCI, and AD) are shown in supplemental table 2, and test set demographics between amyloid groups (CN-Aβ(−), CN-Aβ(−) PITT, CN-Aβ(+)) are shown in supplemental table 3. Cognitive groups differ by sex, chronological age, and education. The CN-Aβ(−), CN-Aβ(−) PITT, and CN-Aβ(+) groups do not differ by chronological age but do differ by education.

3.1. Brain age model prediction: training and independent validation sets

In our training set, our model accurately predicted brain age (r(756) = 0.94, p<0.002; R² = 0.88,; and mean absolute error (MAE) = 4.9 years) (Supplemental Figure 1). We also show that this model does not violate any assumptions (Supplemental Figure 2). In the ADNI CN-Aβ(−) independent validation set, our model also accurately predicted brain age (r(50) = 0.64, R² = 0.42, and MAE = 3.7 years). The model also accurately predicted brain age in the entire test set (r(490) = 0.60; R2 = 0.36; and mean absolute error (MAE) = 4.65 years). The voxel-wise coefficients of the model that predict chronological age are shown in Supplemental Figure 3 and are also in a supplementary file that can be visualized with imaging software.

3.2. Multivariable linear regression model between brain age and chronological age with group effects

For the Aβ(−) trained model results, diagnostic group (CN, EMCI, LMCI, and AD) were significantly associated with BA even after adjusting for race, sex, and education (F(3, 487)=62.3, p<0.0001, Figure 1, Table 1). Pairwise post-hoc group comparisons identified significant differences between groups (Supplemental Table 4): 1) the CN had a lower brain age compared to EMCI, LMCI, and AD; 2) EMCI had a greater association between chronological and brain age compared to LMCI and AD [age by group interaction]; and 3) LMCI had a lower brain age compared to AD. For the Aβ(−) trained model results, CN subgroups (CN-Aβ(−), CN-Aβ(−) PITT, CN-Aβ(+)) were significantly associated with BA even after adjusting for race, sex, and education (F(2, 131)=3.3, p=0.04, Figure 1, Table 1). The CN-Aβ(−) PITT had a lower brain age compared to CN-Aβ(+), but there were no differences between CN-Aβ(−) and CN-Aβ(+) (Figure 1).

Figure 1 -. Multivariable linear regression for brain age models show benefits of Aβ(−) training:

Plots for the test group regression lines of brain age over chronological age for both the Aβ insensitive and Aβ(−) trained models are shown for AD diagnostic groups (top) and amyloid status in the CN group (bottom). An identity line is also provided (dotted black). The Aβ(−) trained model shows incremental differences between diagnostic groups and CN subgroups following AD progression while the Aβ insensitive model does not.

Table 1 – Aβ(−) trained brain ages show significant differences between AD diagnostic groups and amyloid status in CN participants:

Results of the multivariable linear regression analysis of the Aβ(−) trained model are shown for cognitive diagnostic groups and amyloid status as a sub-analysis of the CN group. The reference test group was CN for comparison between diagnostic groups and CN-Aβ(−) PITT for CN subgroups. (CA = chronological age)

Term	Estimate	Std Error	t Ratio	p-value	Lower 95%	Upper 95%
Cognitive diagnostic groups (CN as reference)
Intercept	35.291579	2.243782	15.73	<.0001	30.882867	39.70029
CA	0.5093958	0.029011	17.56	<.0001	0.4523937	0.5663979
EMCI	1.020452	0.467582	2.18	0.0296	0.1017208	1.9391832
LMCI	4.2385508	0.551762	7.68	<.0001	3.1544176	5.322684
AD	6.7979684	0.572472	11.87	<.0001	5.6731422	7.9227945
Amyloid status in CN group (CN-Aβ(−) PITT as reference)
Intercept	34.463869	4.644971	7.42	<.0001	25.274351	43.653388
CA	0.5050906	0.060237	8.39	<.0001	0.3859185	0.6242627
CN-Aβ(−)	1.0575062	0.781668	1.35	0.1784	−0.48893	2.6039429
CN-Aβ(+)	1.9821045	0.781679	2.54	0.0124	0.4356468	3.5285621

For the Aβ insensitive model results, diagnostic group (CN, EMCI, LMCI, and AD) was also significantly associated with BA (F(3, 487)=6.9, p<0.0001, Figure 1, Supplemental Table 5). However, post-hoc group comparisons showed that (Supplemental Table 6) EMCI had greater brain age compared to CN, LMCI, and AD. Although there were significant differences between CN and EMCI, incremental differences between stages did not follow AD progression (CN to EMCI to LMCI to AD). In addition, there were no significant differences between CN subgroups (Supplemental Table 5, F(2, 131)=0.4, p = 0.68).

3.3. Cross-validated simple logistic classifier

The baseline model (ZeroR), which predicts groups by choosing the most common category, had an accuracy of 42%, sensitivity of 21%, and specificity of 80%. We found that chronological age alone has an accuracy of 32%, sensitivity of 24%, and specificity of 82% with an AUC of 0.61. We found that brain age alone improves upon this with an accuracy of 41%, sensitivity of 34%, and specificity of 84% with an AUC of 0.66. Finally, those two features together had an accuracy of 42%, sensitivity of 42%, and specificity of 85% with an AUC of 0.71. We have plotted receiver-operating characteristic (ROC) curves in supplemental figure 5. Using brain age from the Cole model, we found an accuracy of 17%, sensitivity of 19%, and specificity of 80% with an AUC of 0.56. Using brain age from the Cole model combined with chronological age, we found an accuracy of 32%, sensitivity of 22%, and specificity of 82% with an AUC of 0.62.

4. Discussion

We trained a brain age model on individuals without significant amyloid pathology to improve the utility of brain age as a potential biomarker in aging and AD. Our model predicted brain age with similar accuracy as compared with previous brain age models (Franke and Cole 2017, Beheshti 2018), both in cross-validation and in the independent test set. The resulting brain ages significantly distinguished between diagnostic stages of AD (CN, EMCI, LMCI, and AD).

While our model distinguished between all stages of AD diagnoses, later stages showed increasing differences in BA over CA, suggesting that later AD progression results in exacerbated structural changes (as supported by Jack 2013, Sperling 2011).

The slope of the EMCI line was significantly higher than those of LMCI and AD, suggesting that in this stage specifically, BA reflects the greatest extent of structural change over time relative to other diagnostic stages. The other stages with similar slopes may reflect incrementally increased disease burden or diminished reserve rather than “accelerated aging,” although a longitudinal study would be needed for further interpretation.

For the cognitively normal groups, our model was able to significantly distinguish between the CN-Aβ(−) PITT and CN-Aβ(+) groups, but not the CN-Aβ(−) and CN-Aβ(+) groups. This may be attributed to the Pittsburgh cohort being recruited from community dwelling older adults for a normal aging study, whereas the ADNI cohort has been recruited from ADRCs, which may include individuals with subjective cognitive decline or other factors not accounted for in exclusion criteria, which may warrant additional study.

To our knowledge, our model demonstrates the greatest incremental differences in BA over CA between diagnostic stages of AD disease progression. In addition, it is the first to consider amyloid status in defining the BA prediction model. Notably, a prevailing amyloid insensitive brain age model was not able to correctly order the diagnostic stage test groups or distinguish between any CN subgroups. Although comparisons between additional amyloid insensitive models are warranted, these preliminary results show strong potential for the consideration of amyloid status in training of brain age models.

In addition, our results demonstrate that there is potential clinical utility of machine-learning brain age models in the monitoring of AD. When considering that MRI is relatively inexpensive and non-invasive relative to PET and is commonly obtained in cases of subjective cognitive concern without objective memory impairment, more developed models may offer benefits in tracking disease progression and informing decision making regarding PET imaging. Our two-feature (chronological age and brain age) simple logistic classifier was capable of predicting groups, indicating their capacity as predictive features. We noted that brain age improved classification of groups with chronological age.

One limitation of this study is the diminished correlation coefficient in the independent validation set (r = 0.64, compared to r = 0.94 in the training set). The diminished value may be explained by the lower number of participants in the test set (51 vs. 757 in the training set) and the restricted age range of the test sample (60–85 years compared to 20–85 in the training sample), which reduced the total variability that can be explained by the model (Bland and Altman 2011). We also did not evaluate longitudinal changes in cognitive function or amyloid positivity – future longitudinal studies are needed to evaluate these associations.

Regardless, the MAE of our independent validation set is comparable to previously published brain age models (Franke and Cole 2017), which is a better indicator of model accuracy. Furthermore, a prior study demonstrated a similar diminished correlation coefficient and preserved MAE (Beheshti 2018) with a similar age range. Another limitation of this study is the sparsity of participants between the ages of 45–55 as compared to other age groups. This is expected, as amyloid-PET is not often acquired in those younger than 60.

5. Conclusions

Our Aβ(−) trained model performed superior to a contemporary Aβ insensitive model in both fitting BA for CA and distinguishing between groups of different stages of AD progression. Overall, incorporation of amyloid status in brain age prediction models may improve model performance and the utility of brain age as a biomarker of aging and AD.