Anatomically interpretable deep learning of brain age captures domain-specific cognitive impairment

Significance

The phenotypic age of the human brain, as revealed via deep learning of anatomic magnetic resonance images, reflects patterns of structural change related to cognitive decline. Our interpretable deep learning estimates that the brain ages more accurately than any other approaches to date. Furthermore, compared to chronological age, our inferred brain ages are significantly more strongly associated with early signs of Alzheimer’s disease. Maps conveying the importance of each brain region for estimating brain age reveal differences in patterns of neurological aging between males and females and between persons with and without cognitive impairment. These findings provide insight into early identification of persons at high risk of Alzheimer’s disease.

Abstract

The gap between chronological age (CA) and biological brain age, as estimated from magnetic resonance images (MRIs), reflects how individual patterns of neuroanatomic aging deviate from their typical trajectories. MRI-derived brain age (BA) estimates are often obtained using deep learning models that may perform relatively poorly on new data or that lack neuroanatomic interpretability. This study introduces a convolutional neural network (CNN) to estimate BA after training on the MRIs of 4,681 cognitively normal (CN) participants and testing on 1,170 CN participants from an independent sample. BA estimation errors are notably lower than those of previous studies. At both individual and cohort levels, the CNN provides detailed anatomic maps of brain aging patterns that reveal sex dimorphisms and neurocognitive trajectories in adults with mild cognitive impairment (MCI, N = 351) and Alzheimer’s disease (AD, N = 359). In individuals with MCI (54% of whom were diagnosed with dementia within 10.9 y from MRI acquisition), BA is significantly better than CA in capturing dementia symptom severity, functional disability, and executive function. Profiles of sex dimorphism and lateralization in brain aging also map onto patterns of neuroanatomic change that reflect cognitive decline. Significant associations between BA and neurocognitive measures suggest that the proposed framework can map, systematically, the relationship between aging-related neuroanatomy changes in CN individuals and in participants with MCI or AD. Early identification of such neuroanatomy changes can help to screen individuals according to their AD risk.

Although chronological age (CA) reflects disease risk, the rate of aging varies across individuals, organs, tissues, and clinical conditions (1). Because CA does not capture this variation well, there is interest in estimating biological age to predict morbidity (2, 3). Among typically aging adults, in the absence of any clinical indications, biological age is expected to equal CA, on average (4). Neuroanatomic biological age inferred from MRI, henceforth referred to as brain age (BA), can quantify disease-related changes in aging and associated increases in mortality risk (5, 6). Thus, reliable BA estimators can help to stratify individuals according to disease risk (7, 8). The difference between BA and CA, known as age gap (AG), conveys whether aging is faster or slower than expected (9, 10). In clinical cohorts, improving BA estimates can translate into better estimates of participants’ deviations from typical aging (11, 12). For example, BA has the potential to become an affordable and noninvasive preclinical indicator of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) (13) due to the strong association between BA and dementia risk (14, 15).

Deep learning (DL) methods can estimate BA by learning to estimate cognitively normal (CN) subjects’ CAs from MRIs of their brain, while minimizing the mean absolute error (MAE) between BA and CA (16). Compared to other approaches, DL typically yields better BA estimates (17). However, its inherent black-box nature hinders the interpretability of its feature attribution (18), since the relative utility of regional brain features for BA estimation by DL methods is unknown. Furthermore, many DL estimators of BA are inaccurate and lack generalizability to cohorts not encountered during DL training. To address these shortcomings, we introduce an interpretable three-dimensional (3D) convolutional neural network (CNN) to estimate BA from T ₁-weighted brain MRIs. To provide neuroanatomic interpretability, MRI feature attribution is achieved through saliency maps. These allow one to identify structural brain patterns of CN aging that reflect regional and sex-specific variations in neuroanatomic features reflecting BA. 3D-CNN generalizability to new cohorts is also illustrated. The translational potential of this study is reflected in the associations between estimated BAs and neurocognitive measures of CI.

Results

Neuroanatomic Patterns of Aging.

We use an interpretable 3D-CNN framework to estimate the BAs of 650 CN adults (age range: 18 to 88 y; 325 males) from the Cambridge Centre for Aging and Neuroscience (CamCAN, Fig. 1 A and C ). BAs were also estimated in 359 participants with AD dementia (age range: 55 to 92 y; 198 males) and in 351 participants with MCI due to AD (age range 55 to 89; 230 males) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI, Fig. 1A ). Among participants with MCI, 54% were diagnosed with dementia within 11 y from the acquisition of MRIs analyzed in this study. We generated 3D-CNN saliency maps of each participant’s brain to determine how the 3D-CNN weighs each MRI voxel (Fig. 1 B and D ). Saliency maps can help to identify brain locations whose MRI features are weighted more heavily during age estimation (Fig. 2 and SI Appendix, Figs. S1–S12). Using this strategy, we mapped CI-related aging patterns and studied their variation across sexes, brain regions, and subjects, as well as their association with neurocognitive outcome (Fig. 1E ).

Fig. 1.

Overview of BA estimation by an interpretable 3D-CNN. (A) Proportions of participants in the aggregate dataset (ADNI, UKBB, CamCAN, and HCP), where each human symbol represents ∼300 participants. (B) $T_1$ -weighted MRIs were skull-stripped and 3D saliency probability maps were generated from 3D-CNN output for each subject. (C) Prior to BA estimation using the 3D-CNN, participants were split by sex and assigned randomly into training and test sets. MAE was used to evaluate 3D-CNN performance from BA estimation results for test sets. The test set’s CA histogram is displayed in an inset. (D) The 3D-CNN’s input consists of $T_1$ -weighted MRIs, and its output are BA estimates. Saliency maps are extracted from 3D-CNN output after training. A dropout rate of 0.3 is used in all dropout layers, and a ReLU activation function is used in all convolutional and dense layers. x _i is the feature map for input i and w _i is its weight. (E) Sample sizes for participants with neurocognitive measures.

Fig. 2.

Comparison of brain saliency maps across sexes and diagnoses. (A) Sex-specific mean saliency maps (P _M , P _F ) and the sex dimorphism map Δ P = (P _M  − P _F )/[(P _F  + P _M )/2] of CN participants. In all cases, canonical cortical views (sagittal, axial, and coronal) are displayed in radiological convention. Higher saliencies (brighter regions) indicate neuroanatomic locations whose voxels contribute more to BA estimation. Regions drawn in red have higher saliencies in males (P _M  >  P _F ); the reverse (P _F  >  P _M ) is true for the regions drawn in blue. (B) Canonical views of the sex dimorphism map Δ P for CN participants. Sex-specific deviations of Δ P from its mean across sexes are expressed as percentages of the mean. Red indicates that Δ P _M  >  Δ P _F , i.e., males have higher saliency; blue indicates the reverse (Δ P _F  >  Δ P _M ), i.e., females have higher saliency. (C) Like (A), for the comparison between CN participants and participants with CI, where Δ P = (P _{C I}  − P _{C N} )/P _{C N} ; red indicates P _{C I}  >  P _{C N} , blue indicates P _{C N}  >  P _{C I} . (D) Like (B), for the saliency difference Δ P between CN and CI participants. Images are displayed in radiological orientation convention (the right hand side of the reader is the left hand side of the participant and vice versa).

Our results in CN participants (Fig. 2 A and B ) reveal typical neuroanatomic patterns of aging, including ventricular enlargement, atrophy of frontal, temporal, and hippocampal cortices, and cortical thinning. Cortical features are weighted differently across sexes (Fig. 2 A and B ), which suggests that males’ BA estimation is particularly reliant upon Sylvian fissure widening, ventricular enlargement, and cingulate cortex atrophy. Males’ BA estimation is also weighted more heavily by features of the lateral temporal lobe and dorsolateral frontal lobe in the right hemisphere, a notable lateralization effect. By contrast, females’ saliencies are higher in posterior and medial occipital regions (except the left calcarine sulcus), in the inferior and medial aspects of the parietal lobes, in the supramarginal gyrus and adjacent parietal structures, in the callosal sulcus, in the pars triangularis of the right inferior frontal gyrus, and in posterior insular regions (Fig. 2 A and B ). In females, on average, white matter is weighted more heavily than gray matter when estimating BA.

Fig. 2 C and D compares subject-wise average saliency maps according to the cognitive status (CN vs. CI). This comparison reveals brain features upon which the 3D-CNN relies more when estimating age according to cognitive status. For this reason, such features may reflect how CI modifies regional brain aging. Many structures salient in CN aging are in the cortical gray matter and include the dorsolateral aspect of the right frontal lobe, the lateral aspect of the right temporal lobe, the posterolateral aspect of the right occipital lobe, as well as pericallosal regions in both hemispheres (Fig. 2 C and D ). Cerebral white matter is more salient in aging with CI than in CN aging (Fig. 2C ), as is the brainstem, medial aspects of the temporal lobes (including parahippocampal and fusiform gyri), and the caudal portions of the anterior cingulate gyri (Fig. 2D ). Appreciable lateralization of saliencies is noted when comparing CN participants to participants with CI, and the lateralization pattern is similar to that obviated by the sex comparison (Fig. 2B ). Involved are lateral temporal areas, the angular and supramarginal gyri, middle cingulate cortex, parahippocampal areas, and both medial and dorsolateral prefrontal cortices.

Associations with Neurocognitive Endophenotypes.

The ability of estimated BA to capture neurocognitive endophenotypes was contrasted to that of CA. This was achieved by comparing Spearman’s correlations r _S between each age (BA and CA) and every neurocognitive measure of CN aging (Fig. 3 and SI Appendix, Tables S1–S5). For all neurocognitive measures, significant r _S values reflect typical aging effects on neurocognitive function (worse performance is correlated with older age). As expected, among CN participants, BA and CA reflect cognition to similar extents. For example, among CamCAN CN participants (Fig. 3A and SI Appendix, Table S1), older BA and CA are correlated with worse performance on word finding (picture priming), motor learning (force matching), motor response time [choice and simple response time (RT) tasks], face recognition (Benton’s unfamiliar face recognition, famous faces test), Cattell’s fluid intelligence, emotional memory, and visual short-term memory (VSTM) measures. In the ADNI CN cohort (Fig. 3B and SI Appendix, Table S2), no neurocognitive measure examined is significantly more correlated with BA than with CA.

Fig. 3.

Correlations between neurocognitive measures and both estimated BA and CA. Results are depicted for two independent test sets: CamCAN and ADNI. (A) displays CN participants from CamCAN, (B) displays CN participants from ADNI, (C) displays results only for participants with MCI, and (D) displays results for participants with either MCI or AD. For each independent test set, the sample size for each neurocognitive measure is listed below the measure name. Bar charts depict Spearman’s correlations r _S (along x) between BA (green) or CA (red) and each neurocognitive measure (along y). Bars are contoured in black if r _S is significant. Error bar widths equate to one SE of the mean. For each neurocognitive measure, the corresponding bar pair is annotated with Fisher’s z statistic. Asterisks indicate neurocognitive measures for which the difference in Spearman’s correlations r _S (B A)−r _S (C A) is significant.

Across participants with CI, BA is significantly more correlated than CA with neurocognitive measures. In participants with MCI (Fig. 3C and SI Appendix, Table S3), older BA (but not CA) is significantly correlated with worse scores on all measures of neurocognitive function examined, except 1) delayed verbal recall and learning on the Rey auditory verbal learning test (RAVLT), 2) delayed word recall measured by the AD assessment scale question 4 (ADAS Q4), and 3) logical memory. For the clinical dementia rating sum of boxes (CDR-SB) and the functional abilities questionnaire (FAQ), the difference in correlations between BA and CA is significant and BA outperforms CA in its ability to reflect neurocognitive function. In participants with AD, no significant correlations exist between BA and any neurocognitive measure apart from FAQ scores ( SI Appendix, Table S4). Nevertheless, older CA is correlated with poorer delayed verbal memory (RAVLT forgetting). By contrast, among all participants with any type of CI (whether MCI or AD), BA (but not CA) is significantly correlated with all measures except delayed verbal recall (RAVLT, ADAS Q4) and logical memory (Fig. 3D and SI Appendix, Table S5). The difference in correlations between BA and CA is significant for the CDR-SB, mini-mental state exam (MMSE), RAVLT immediate recall (IR), and FAQ. When separating participants with CI by apolipoprotein E4 (APOE4) status, BA is not more correlated with any neurocognitive measure in carriers compared to noncarriers. The omnibus effect of a logistic regression accounting for all interactions between AG, CA, and sex is significant (χ ₃₄₃ ² = 29.500, P < 0.001). AGs are significantly and positively associated with MCI participants’ probability of conversion to AD (β = 1.417, t ₃₄₃ = 2.240, P = 0.025). The only significant interaction is between AG and sex (β = −1.121, t ₃₄₃ = −2.129, P = 0.033), i.e., MCI females with more negative AGs and MCI males with more positive AGs are significantly more likely to convert to AD. When including all interactions, the omnibus effect of the regression that predicts time to conversion is significant if AG is included as the predictor (R ² = 0.065, F _{8, 181} = 2.880, P = 0.007) but not if AG is excluded (R ² = 0.012, F _{4, 185} = 1.790, P = 0.151).

3D-CNN Benchmarking and Evaluation.

We compare our 3D-CNN to an award-winning (19) state-of-the-art model, the simple fully convolutional network (SFCN) of Gong et al. (20, 21), by replicating its training, validation, and benchmarking. The SFCN was pretrained on 5,698 UK Biobank (UKBB) subjects, whereas our 3D-CNN was trained on 4,681 participants (2,513 females; age range: 22 to 95 y) aggregated across the UKBB, Human Connectome Project-Aging (HCP-A), Human Connectome Project-Young Adult (HCP-YA), and ADNI. In the testing set, our model’s MAE between BA and CA is 2.41 y for males and 2.23 for females ( SI Appendix, Fig. S13 A and B ). The coefficient of determination R ² is 0.96; the correlation coefficient r is 0.98. Across all external testing sets (UKBB, CamCAN, AD, and MCI), our model has a higher R ² than the SFCN ( SI Appendix, Table S6).

On identical UKBB data (N = 518), the 3D-CNN achieves MAEs of 2.27 y (males) and 2.31 y (females) ( SI Appendix, Fig. S14 A and E ), while the SFCN achieves an MAE of 2.14 y across both sexes ( SI Appendix, Fig. S14 Q and R ). In the independent CamCAN CN cohort, the SFCN’s MAEs are 9.90 y (males) and 9.17 y (females) ( SI Appendix, Fig. S14 Q and R ). By contrast, our 3D-CNN achieves MAEs of 4.71 y (males) and 3.01 y (females) ( SI Appendix, Fig. S14 B and F ). During pretraining, the SFCN yields an MAE within 2% of the 3D-CNN’s. However, in the independent test cohort of CN participants, our MAEs are 42% lower than the SFCN’s in the same cohort. The SFCN yields MAEs of 7.72 y (males) and 7.50 y (females) for participants with MCI and 8.24 y (males) and 8.65 y (females) for participants with AD ( SI Appendix, Fig. S14 Q and R ). By contrast, our 3D-CNN model achieves an MAE of 5.26 y (males) and 4.33 y (females) for participants with MCI ( SI Appendix, Fig. S14 C and G ) and 6.48 y (males) and 5.98 y (females) for participants with AD ( SI Appendix, Fig. S14 D and H ). Compared to the SFCN ( SI Appendix, Fig. S14 Q and R ), the 3D-CNN yields significantly larger mean AGs for A) females with MCI (t ₁₄₄ = 6.595, P <  0.001), B) males with AD (t ₁₉₅ = 4.710, P < 0.001) and C) females with AD (t ₁₆₂ = 6.200, P < 0.001). The 3D-CNN also yields significantly larger AG variances in participants with AD (males: F _{197, 197} = 1.857, Pitman’s t ₁₉₆ = 4.440, P <  0.001; females: F _{162, 162} = 2.493, Pitman’s t ₁₆₁ = 6.006, P <  0.001). Compared to the 3D-CNN, the SFCN yields significantly larger AG variances for the following CN groups: A) UKBB males (F _{796, 796} = 1.137, Pitman’s t ₇₉₅ = 12.967, P <  0.001); B) UKBB females (F _{796, 796} = 1.097, Pitman’s t ₇₉₅ = 9.034, P <  0.001); C) CamCAN females (F _{309, 309} = 7.576, Pitman’s t ₃₀₈ = 21.082, P <  0.001). As expected, the 3D-CNN’s mean AG is ∼75% larger in participants with CI than in CN participants ( SI Appendix, Fig. S14 I–P ), possibly reflecting faster brain aging in the former. The BA estimation parameters of the 3D-CNN and SFCN, evaluated without fine-tuning, are compared in Fig. 4 and SI Appendix, Table S6. The 3D-CNN has shorter execution times (ETs) and fewer trainable parameters, reflecting lower complexity ( SI Appendix, Table S6 and Fig. 4). As SI Appendix, Table S6 and Fig. S14 suggest for participants with CI, our CNN yields higher R ² and lower MAEs than the SFCN.

Fig. 4.

Radar plots of sex-specific MAEs and performance parameters. Radar plots of MAE, R ², and performance parameters (average ET and the number of trainable parameters) according to sex and diagnostic status (CN: UKBB, CamCAN; MCI or AD: ADNI). The SFCN of Gong et al. (20, 21) (purple) is compared to our 3D-CNN (blue). To facilitate simultaneous comparison, all values are normalized to range from 0 to 1, where the maximum value in each measurement was rescaled as 1 and 0 remained as 0.

Discussion

Significance.

While biological age can be computed for many phenotypic traits, BA summarizes MRI-derived neuroanatomic profiles using one number. This highlights both the appeal and caveats of this measure. Although straightforward to grasp, BA (as defined here) does not capture the nuances and complexity of brain aging. Nevertheless, with cautious interpretation, BA could assist the diagnosis and prognosis despite its limitations. Early screening for CI can help to monitor and improve the welfare of aging adults (22). Although positron emission tomography (PET) can aid the diagnosis of AD at the preclinical and prodromal stages (23), this technique is expensive, involves specialized tracers, and exposes participants to radiation (24). By contrast, MRI is noninvasive, more affordable, and safer. Thus, MRI-derived BAs that capture neurocognitive decline (25) could become affordable and noninvasive preclinical measures of CI risk (26).

The correlations of neurocognitive measures with our estimated BAs are, in many cases, significantly stronger than with CA, suggesting that our BAs better reflect neurocognitive functioning. These correlations are critical because one potential utility of BA estimation is to facilitate the early identification of persons at high risk of MCI and AD. AGs are predictive of AD conversion risk, as others reported (27 –29). BA is not correlated with neurocognitive function in participants with AD, with the exception of informant-rated functional ability. One possible reason is that the 3D-CNN was trained on CN adults rather than participants with CI. Another reason could be that correlations are more difficult to detect due to lower statistical power (smaller sample size) in participants with AD compared to CI participants. Across persons with CI of any severity, BA (but not CA) is significantly correlated with measures used routinely (30) to screen for (or to diagnose) CI, including MCI (Fig. 3 and SI Appendix, Tables S4 and S5). Thus, our contributions can help to understand how CI-related neurocognitive changes within specific functional domains reflect neuroanatomic features that modify regional BAs.

Sex Differences in Anatomic Brain Aging.

Of note, for patient-tailored profiling, our approach can generate subject-specific brain saliency maps reflecting individual neuroanatomic patterns of brain aging. Anatomic interpretability of BA is important because 1) brain regions age differently, 2) neuroanatomic alterations with age may reflect distinct disease processes paralleled by BA (31), and 3) individual neuroanatomic deviations may parallel neurocognitive endophenotypes. Sex differences in saliency confirm findings on the contributions of age to sex dimorphism in the pre- and post-central gyri (32, 33) and the pars triangularis of the left inferior frontal gyrus (34). Males, who are at higher risk of motor impairment due to Parkinson’s disease (35), exhibit greater saliency in the primary motor cortex. Males’ BA estimation relies more on the crowns of gyri on the lateral aspects of the frontal lobes, whereas females’ BA estimation relies more on the troughs of sulci. These findings confirm prior reports on sex differences in older adults’ cortical gyrification (36). Males’ saliencies are higher along ventricular boundaries, indicating that BAs are disproportionately predicated upon ventricular enlargement in men, as reported elsewhere (37). The right hemisphere’s higher saliency in males is consistent with their lateralization of language function (38) and with lateralization trends in old age (39). Thus, in females, typical cortical aging may be relatively slower in the right hemisphere. By contrast, on average, most occipital and medial parietal areas exhibit age-related neuroanatomic patterns that are more salient in males. Males also have higher saliency in superior parietal and frontal regions, reflecting smaller gray matter volumes (40). By contrast, females have higher saliency at the occipital poles and in occipitoparietal regions, reflecting smaller gray matter volumes in these regions (40). Females’ saliencies are higher across inferior parietal regions, where the cortex is thicker than in males (41). Thus, our approach to neuroanatomic saliency mapping can identify sex differences in the neuroanatomy of cortical aging.

Anatomy Changes According to Neurocognitive Status.

Our interpretable 3D-CNN framework captures neuroanatomy changes related to both CN aging and aging with CI. In the case of CN aging, the estimated BAs of CN participants in our two independent samples (CamCAN and ADNI) are correlated with neurocognitive measures reflecting typical aging (e.g., motor learning, multitasking, and word finding). In ADNI CN participants, no significant associations were found between neurocognitive measures and either CA or BA. This was expected, as ADNI cognitive measures are sensitive to CI rather than to CN aging (42). In the case of CI, SI Appendix, Fig. S14 I–P confirms that participants with either MCI or AD have AGs considerably larger than those of sex- and age-matched CN adults (43), mostly due to older-than-expected brains (BA > CA). Atrophy of the parahippocampal gyrus is a strong structural correlate of MCI and AD (44); our 3D-CNN’s greater reliance on this structure during BA estimation reflects this (Fig. 2C ). Similarly, saliency differences between CN and CI participants are greater in parietal, occipital, and temporal cortices (Fig. 2D ), whose atrophy is greater in participants with CI (45) and whose burdens of amyloid β plaques and τ neurofibrillary tangles are typically higher in AD (46). The brainstem, which is affected by amyloid deposition early during AD, is more salient in participants with CI than in CN adults (47). Comparison of the cortical patterns in Fig. 2 B and D indicates that saliency differences between sexes are largely paralleled by saliency differences across cognitive statuses (CN vs. CI). This may reflect females’ higher risk for AD and supports the hypothesis according to which their higher risk is paralleled by faster cortical aging. Comparison of CN and CI cohorts suggests that the SFCN underestimates mean AG in the latter group and that the expected accuracy of BA estimation is lower for participants with CI. These findings highlight the importance of an accurate BA estimator when studying diseased populations. Some cortical structures that atrophy far more in CI than in CN aging are more salient in the latter (blue regions in Fig. 2D ). This may reflect the fact that the 3D-CNN was trained on a CN adult cohort. During training on this cohort, our 3D-CNN likely relies on features whose variance is moderate in CN aging. When estimating the BAs of participants with CI, however, these features exhibit far greater variability. This may cause their relative saliency to decrease, such that the saliency difference Δ P between CN and CI aging is negative in such regions. Thus, although features with negative Δ P values can be useful for understanding how BA estimation relies on CI-related neuroanatomy features, the negative sign of Δ P must be interpreted cautiously.

Comparison to Other Methods.

Our 3D-CNN alleviates major limitations of other approaches. The quantitative comparison below focuses on the SFCN because this open-source approach performed best in a competition (19) for which both training and testing data are available.

Accuracy.

Our 3D-CNN estimates BA more accurately than the state of the art regardless of whether accuracy is quantified using MAE or R ². In the test set, our model yields an MAE of ∼2.3 y; this is ∼1 y less than the SFCN, which is the second best. Other (published) BA estimators have MAEs that are even higher than that of the SFCN on their testing data. Presumably, since our MAE is ∼2.3 y, these estimators also perform more poorly than ours. However, we could not ascertain this because we did not have access to the testing sets on which others estimators were benchmarked. These estimators include a best linear unbiased predictor (MAE   ≃  3.3 y) (48), a 3D residual neural network (3D-RNN, MAE   ≃  3.3 y) (49), a graph CNN (MAE ≃ 4.6 y) (50), Gaussian process regression (MAE ≃ 4.1 y) (16), support vector regression synergized with a random forest classifier (MAE ≃ 3.5 y) (51), and a 3D-DenseNet (MAE   ≃  3.3 y) (52).

On the testing set, our model yields R ² ≃ 0.96 and r ≃ 0.98. By contrast, the SFCN model yields R ² ≃ 0.92 and r ≃ 0.96 (16, 20, 21). During testing, other (published) BA estimators achieve even lower R ² than the SFCN. These include Gaussian process regression (R ² ≃ 0.91) (16), a 3D-RNN (R ² ≃ 0.90) (49), a graph CNN (R ² ≃ 0.87) (50), and a 3D-DenseNet (R ² ≃ 0.85) (52). Our R ² is also higher than that of a BA estimator that used an optimized SFCN (53) with R ² = 0.94. These comparisons suggest that even high R ² can involve undesirably large MAE, such that it can be useful to consider both measures when evaluating accuracy.

In all females with CI and in AD males, our AGs are significantly larger than those estimated by the SFCN. As expected, the 3D-CNN’s estimates of these subjects’ CAs are consistently larger than their true CAs. Because CI involves more brain aging, this suggests that the 3D-CNN captures CI better than the SFCN. Females are at higher risk for AD and exhibit faster decline than males (54). Females already have a larger mean AG in the MCI stage, whereas this is not the case for males until the AD stage. Thus, our model captures known sex differences in AD risk.

Variances in AG between the 3D-CNN and SFCN are significantly different for the UKBB cohort even though F = σ _{S F C N} ²/σ _{C N N} ² ≃ 1, which usually implies lack of significant differences in variance. This finding can be explained by our use of Pitman’s variance ratio test, which is justified here because the variances being compared pertain to correlated samples (CNN- and SFCN-computed AGs measured for the same cohort). Because the 3D-CNN and SFCN were both trained on UKBB CN participants, the abilities of these methods to estimate BA for new UKBB participants are likely better (and therefore more similar) than their ability to estimate BA for participants from altogether new cohorts. This similarity may explain the strong correlation r of UKBB BAs across the two methods (females: r = 0.989; males: r = 0.990). The dependence of Pitman’s t on r (see Methods) satisfies t ∼ (1 − r ²)^−1/2. A Maclaurin series expansion indicates that t → ∞ as r → 1. Thus, Pitman’s t is large when r ≃ 1 even when σ _{S F C N} ²/σ _{C N N} ² ≃ 1. This explains our power to detect even moderate differences between σ _{S F C N} ² and σ _{C N N} ² in UKBB CN participants.

Complexity.

Model complexity was quantified using mean ET and the number of trainable parameters in the model. By both measures, our 3D-CNN’s execution complexity is lower than that of previous approaches. For example, the 3D-CNN features a ∼10 times shorter ET compared to the SFCN (20, 21) and ∼4 times fewer trainable parameters. The model of Leonardsen et al. (53), which is based on the SFCN (20, 21), has more trainable parameters and is more challenging to fine-tune. The 3D-DenseNet (53) has ∼7 million trainable parameters (compared to our 682,881) and requires extensive fine-tuning on new validation datasets via grid searches for optimal hyperparameters.

Interpretability.

Lee et al. (52) use a 3D-DenseNet to compute saliency by covering the brain with occlusion masks of size 11³ mm³ = 1, 331 mm³. According to these authors, their saliencies correlate with PET-mapped amyloid β and τ burdens. However, for participants with CI, the anatomic patterns of brain aging mapped by Lee et al. are broadly similar to ours (Fig. 2 C and D ) across similar age ranges. This suggests the hypothesis that BA saliencies like ours can reflect dAD-related clinical PET findings. He et al. (55) used two-dimensional (2D) occlusions (box size: 32² mm² = 1, 024 mm²) to map saliency, whereas Wood et al. (56) monitored performance and saliency by occluding 3D masks (size: 5³ mm³ = 125 mm³). Our study advances the state of the art by 1) providing voxelwise saliency maps to reveal detailed spatial variability at native MRI resolution (1 mm³), 2) reporting comparisons by sex and cognitive status, and 3) conveying how cognitive status relates to neurocognitive function.

Generalizability.

Most BA estimators are not typically tested across domain-specific neurocognitive measures, whereas our 3D-CNN features unique generalizability to independent cohorts in its ability to capture neurocognitive endophenotypes. Since the R ² values achieved on independent and test data are similar, we surmise that overfitting was largely avoided. Compared to CA, the BA of participants with any type of CI is significantly more correlated with measures of neurocognitive function routinely used as clinical indicators of CI. Other published approaches have rarely been evaluated according to this critical performance benchmark. Because the 3D-CNN was trained on subjects aged 22 to 95 y, its utility extends across the age range of adulthood.

Limitations.

Although we validated the 3D-CNN in cohorts independent from those used for its training, differences in acquisition sequences and scanners across MRIs can affect results (57). Like other dementia diagnosis criteria, ADNI’s have limitations [e.g., a risk of false positive diagnoses (42) that may affect the findings of studies like ours. Additionally, floor effects may affect cognitive measures in participants with AD by attenuating their correlation. Conceivably, our failure to find significant correlations between BA and neurocognitive measures in participants with AD could be due to our lower power to detect small effects in the AD sample, which is smaller (N ≤ 172) compared to the MCI (N ≤ 347) and combined CI (i.e., MCI or AD, N ≤ 519) samples. These nonsignificant correlations, however, are not typically relevant for early CI screening because most participants with severe CI have been already diagnosed by the time brain MRIs are typically acquired. The nonuniform distribution of CAs in our aggregate sample translates into potential training data imbalance and inaccuracy in BA estimates. Nevertheless, our approach is more accurate than others currently available, as reflected by our test set’s MAE and R ², which are the lowest reported to date. Due to the lack of ground truth, there is no consensus on how the interpretability of approaches like ours ought to be evaluated (52, 55, 56, 58).

Methods

Participants and Neuroimaging.

This study was undertaken in adherence with the US Code of Federal Regulations (45 C.F.R. 46) and the Declaration of Helsinki. MRIs analyzed in this study were acquired as part of other studies, with approval from the institutional review boards or similar ethical monitoring bodies at the respective institutions where data had been acquired for ADNI (30) and HCP (59). UKBB efforts were undertaken with ethical approval from the North West Multi-Centre Research Ethics Committee of the United Kingdom. Ethical approval for CamCAN was obtained by the Cambridgeshire 2 (now East of England—Cambridge Central) Research Ethics Committee. Informed written consent was obtained from all participants.

The aggregate dataset consists of 5,851 CN individuals (3,142 females) aged 22 to 95 yr sampled from ADNI (N = 510), HCP-A (N = 508), HCP-YA (N = 1, 112), and UKBB (N = 3, 721; Table 1). The ADNI was launched in 2003 as a public–private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial MRI, positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. MRI acquisition protocols for HCP-A and HCP-YA are described elsewhere (60). For UKBB data, we used preprocessed images generated by a UKBB pipeline whose output included FreeSurfer reconstructions (61).

Table 1.

Participant demographics

			CA (y)				Ratio	FreeSurfer version
Repository		Status	N	Min	Max	μ	σ	M:F	4.3.0	5.3.0	6.0.0	7.1.1
ADNI		CN	510	56	95	75.1	7.2	1:1.17	0	0	260	250
HCP-A		CN	508	36	80	55.8	12.0	1:1.38	0	0	309	199
HCP-YA		CN	1,112	22	37	28.8	3.7	1:1.17	0	1,112	0	0
UKBB		CN	3,721	45	83	62.7	10.1	1:1.13	0	0	3,721	0
CamCAN		CN	650	23	88	54.2	18.6	1:1.00	0	0	0	650
ADNI		AD	359	56	95	75.9	8.0	1:0.83	359	0	0	0
ADNI		MCI	351	55	89	75.2	7.3	1:0.53	351	0	0	0
All		all	7,211	23	95	58.4	10.3	1:1.09	710	1,112	4,290	1,099

Sample size, descriptive statistics (minimum, maximum, mean μ, and SD σ), the male-to-female (M:F) sex ratio, and breakdown by FreeSurfer version used for preprocessing. Demographics are listed for each repository and neurological/cognitive status.

An independent test set of 650 CN participants aged 18 to 88 yr was obtained from CamCAN. Additionally, 408 participants with MCI and 359 participants with AD were obtained from ADNI (Table 1). CamCAN inclusion/exclusion criteria (62) and ADNI eligibility/diagnosis criteria are described elsewhere (30). N = 75 participants with MCI were excluded due to MCI diagnosis being unrelated to AD, leaving 351 participants with MCI (190 converted to AD, 161 did not). Of 524 CI (MCI or AD) participants whose correlations between BA and neurocognitive scores were computed, 307 participants were APOE4 carriers.

Neurocognitive Measures.

We used neurocognitive measures available in CamCAN and ADNI to evaluate the utility of our estimated BAs to capture neurocognitive phenotypes (see SI Appendix, Methods for detailed task descriptions). Thirteen cognitive measures that assess emotional processing, executive function, memory, and motor function were obtained from the CamCAN repository (62). Emotional processing was measured via 1) Ekman’s emotion expression recognition test, 2) the emotional memory test, and 3) the emotional regulation test. Executive function was measured using 1) Cattell’s fluid intelligence test, 2) the hotel test, and 3) a proverb comprehension task. Memory was measured using 1) Benton’s face recognition test, 2) the famous faces test, 3) a picture priming task, 4) the tip of the tongue (ToT) test, and 5) a VSTM task. Motor function was assessed via 1) a force matching task, 2) a motor learning task, 3) a reaction time (RT) “choice” task, and 4) a RT “simple” task.

Nine cognitive measures that assess neural function, cognitive performance, and functional impairment were obtained from the ADNI repository (63). To eliminate systematic variability in FreeSurfer software versions, we limited correlation analysis for the CI cohort to subjects from ADNI₁ only (FreeSurfer v4.3). For neural function, four established dementia rating scales were obtained, including 1) the clinical dementia rating scale—sum of boxes (CDR-SB), 2) the diagnostic ADAS versions 11 and 13, and 3) the MMSE. Cognitive performance was measured via four neuropsychological measures: 1) the RAVLT, 2) delayed recall on the logical memory test, 3) the digit symbol substitution test, and 4) the trail-making test. Functional impairment was measured by the FAQ.

MRI Preprocessing.

Freesurfer’s recon-all function was used to reconstruct and segment $T_1$ -weighted MRIs. This process includes skull-stripping, motion correction, normalization of nonuniform signal intensities, Talairach space transformation, removal of nonbrain tissues, and registration of all subjects’ brains into a common coordinate space (64). FreeSurfer (FS) was used for three reasons: 1) UKBB makes FS reconstructions available; 2) the FS workflow is fully automated and thus convenient; 3) our study involves surface analyses and registrations across native and atlas spaces, which FS facilitates. During FS preprocessing using recon-all, all MRIs were affinely registered to the MNI₃₀₅ atlas. Due to sourcing from several MRI repositories, enhancement of segmentation accuracy differed slightly between cohorts (Table 1). UKBB and HCP-YA reconstructions were enhanced using T ₂-weighted MRIs, while ADNI, HCP-A, and CamCAN were enhanced using fluid-attenuated inversion recovery MRIs.

3D-CNN Architecture.

We constructed a DL regression model using a 3D-CNN whose inputs are FS brain.mgz output files and whose outputs are estimated BAs. The DL architecture was implemented in Python 3.6 using TensorFlow 2.7.0 and executed on a computer with an Intel Core i7 processor (2.2 GHz clock speed) with 16 GB of RAM and a 12 GB NVIDIA Tesla K80 graphical processing unit. The 3D-CNN consists of three convolutional blocks followed by two dense layers. The input matrix size is 82 × 86 × 100. Each convolutional (conv) block has a 3D conv layer, a batch normalization layer, a max-pooling layer, and an optional dropout layer. The filter sizes of the first three (conv) blocks are 64, 128, and 128, respectively. Conv block filter size determines the dimensionality of the output space. The rectified linear unit (ReLU) activation function is applied to all conv and dense layers. The ReLU activation function is defined as g(x)=m a x(0, x) for input x. g(x) can efficiently reduce the likelihood of a vanishing gradient and makes the output more sparse. After the conv blocks, the fourth block consists of one global average pooling layer (used for global average pooling of 3D data), one dense layer, and one dropout layer (dropout rate = 0.3). The resulting feature map, of size 18 × 18 × 18 × 128, is pooled to 128 × 1 and then projected onto the output dense layer, which has one output neuron to estimate BA using regression.

We choose MSE as loss function and use an Adam optimizer (learning rate = 0.001) (65). The advantage of outputting BAs as real numbers rather than assigning them to discrete age bins (20, 21) is that, in the former case, BA outputs are assigned within a continuous domain and range. Due to regression to the mean (66), estimated BAs exhibit a previously documented CA-dependent bias. To alleviate this effect, we use the zero correlation constraint method of Treder et al. (67) to regress out the bias from the BAs of testing set participants. This is done separately for each cohort. Bias-corrected BAs are used for all analyses. CN participants are aggregated from UKBB, HCP-A, HCP-YA, and ADNI. Participants were randomly assigned into training and test sets of sizes equal to 20% of the total sample size (N= 5,851).

3D-CNN Training and Testing.

We optimized the CNN architecture and fine-tuned hyperparameters. 2D-CNNs use 2D kernels to estimate sliding windows across single slices, such that leveraging information from adjacent slices is challenging (68). We therefore chose a 3D-CNN that overcomes this deficit by using 3D kernels to estimate sliding windows for volumetric patches. The latter captures interslice image context and improves the model performance 68. We also included dropout and batch normalization layers because these help to alleviate overfitting (69). Grid and random searches determined suitable hyperparameter values (e.g., batch size, kernel size, weight decay). An n-dimensional grid was defined to map the n hyperparameters and to identify their ranges. We examined all possible 3D-CNN configurations to identify optimal values for each hyperparameter. Since we used MSE as a loss function, we selected a configuration with the lowest loss value (error).

We tested the 3D-CNN on independent cohorts to refine the 3D-CNN architecture, illustrate model generalizability, alleviate data overfitting, and compare the 3D-CNN to other approaches. The testing set was designed to include a random selection of participants from the same cohorts as the training set. To avoid overfitting the 3D-CNN to the training set, we monitored its performance on the testing set. To avoid overfitting on both training and testing sets, we tested our model on two independent cohorts (CamCAN and ADNI) that had not been used for the 3D-CNN design. The latter of these cohorts includes participants with a range of cognitive statuses (CN, MCI, or AD).

After computing AGs for identical samples using both our 3D-CNN and the SFCN, we performed Welch’s t-tests for paired samples with unequal variances to compare the mean AGs obtained using the two methods. AG variances were compared using Pitman’s variance ratio test for correlated samples, whereby F = σ ₁ ²/σ ₂ ², Pitman’s $t_{N-2}=[(F-1)\sqrt{N-2}]/[2\sqrt{F(1-r^2)}]$ , and r is the correlation of AG _{S F C N} with AG _{C N N} . The AG variances are σ ₁ and σ ₂, whose subscripts {1, 2} denote the SFCN or CNN, as needed, to satisfy the inequality σ ₁ >  σ ₂.

BA Associations with Sex and Neurocognition.

Each neurocognitive measure m was obtained from CN participiants (ADNI and CamCAN) and from participants with MCI or AD (ADNI). These measures were not normally distributed (Anderson–Darling goodness-of-fit test, SI Appendix, Tables S1–S5), so their Spearman rank correlations r _S (m, B A) and r _S (m, C A) were computed. These correlations were compared using Fisher’s two-sided z-test after multiple comparison correction using the Benjamini–Hochberg procedure (70) (false discovery rate = 0.05). |r _S | and |z| were also calculated for measures whose lower scores indicate better performance. Test statistics and their degrees of freedom, confidence intervals, and effect sizes were tabulated ( SI Appendix, Tables S1–S5). A logistic regression examined whether AG, CA, sex, and their interactions predicted the probability of c o n v e r s i o n from MCI to AD. Another linear regression (independent variables: CA, sex, and their interactions) evaluated the ability of AG to predict the i n t e r v a l between MRI acquisition and AD conversion. AG and its interactions were added to this (reduced) model to examine how AG altered the significance of the regression.

Saliency Maps.

A saliency map is a topographically organized depiction of the visual saliency in an MRI volume V ₀. Here, we extend a saliency approach for 2D-CNNs (71) to the 3D case. For an MRI brain volume V ₀ and a 3D-CNN model with score function S(V), we rank voxels in V ₀ based on their importance to S(V). We consider the linear score model S(V)=w ^T V + b, where the volume V, weight w, and bias b are in one-dimensional (vectorized) forms. Since the 3D-CNN and score function are highly nonlinear functions of V, the linear score model cannot be applied directly. We approximate S(V) at V ₀ using the first-order Taylor series S(V ₀)≃w ₀ ^T V ₀ + b ₀, where w ₀ = ∂S/∂V∣ _{V 0} is the partial derivative of S(V) at V ₀ and b ₀ = b∣ _{V 0} is the bias b at V ₀. The spatial and temporal distributions of saliencies contain unique patterns conveying information about BA.

Saliency Associations with Sex and Neurocognition.

Two distinct workflows were used for volume- and surface-level transforms, respectively, to remove the confounding effects of subject differences in brain shape and size. For v o l u m e-level analysis, each saliency map was nonlinearly registered to the FS fsaverage atlas. To this end, $T_1$ -weighted brain volumes were first registered to the atlas in MATLAB using the imregister function, which applied the transformation from native space to the atlas, as provided by FS. MATLAB’s imregdemons function was used to deform nonlinearly and to map $T_1$ -weighted scans onto the atlas. The transformations above were applied to each subject’s saliency map, resulting in its registration to the atlas. For s u r f a c e-level analysis, saliencies were projected to the native cortical surface. To achieve this, each subject’s saliency was projected onto the cortical mantle as a cortical overlay using a customized algorithm for volume-to-surface mapping (72, 73). Briefly, voxels assigned to the gray matter ribbon by FS were considered. At each vertex of the native mesh for the mid-thickness surface, ribbon voxels were selected within a cylinder that lay orthogonally with respect to the local surface. The cylinder was centered on the vertex; its height and radius were equal to the local cortical thickness. The saliency of ribbon voxels within the cylinder was averaged according to a Gaussian weighted function (full width at half maximum = 4 mm, σ  =  5/3 mm) to compute a mean saliency value at the surface vertex in question. After cortical surface projection, each subject’s saliency overlay was registered from native space onto the atlas. Subjects’ saliency probability overlays were averaged into a cortical map of mean saliency.

For both volume- and surface-level analyses, each saliency map M was operationalized into a saliency probability map P by dividing saliency at each brain location by the sum of all brain saliencies. An average saliency probability map was computed for each sex and cognitive status, yielding P _M for males, P _F for females, P _{C N} for CN adults, and P _{C I} for participants with any form of CI. Both P _{C N} and P _{C I} were computed after averaging across sex effects. Relative sex differences in P were computed as (P _M  − P _F )/[(P _F  + P _M )/2], i.e., as sex-specific deviations from the average across sexes. The relative deviation of participants with CI from CN participants was computed as (P _{C I}  − P _{C N} )/P _{C N} . Relative saliency differences between sexes or diagnostic statuses were mapped after thresholding to include only statistically significant values. For each salience value considered, significance was evaluated using a paired-sample t-test (α = 0.05). Results were corrected for multiple comparisons using the Benjamini–Hochberg procedure (false discovery rate =0.05).

Visualization.

For v o l u m e-level visualization, CN participants’ mean saliency maps were plotted for each sex along the coronal (x), sagittal (y), and axial (z) planes. For each coordinate, maps were generated along planes whose equations were specified by coordinate values of −28 mm, 0 mm, and 28 mm, respectively. In CN participants and participants with CI, the procedure was repeated after averaging across sexes. For s u r f a c e-level visualization, gray matter saliencies were mapped onto the cortex to compare different cortical locations’ relative importance to the 3D-CNN when estimating BA.

Supplementary Material

Appendix 01 (PDF)

Dataset S01 (DOCX)

Data, Materials, and Software Availability

3D-CNN software is available from https://github.com/irimia-laboratory/USC_BA_estimator MRI data are publicly available from ADNI (https://adni.loni.usc.edu/), UKBB (https://www.ukbiobank.ac.uk/), CamCAN (https://www.cam-can.org/), and HCP (https://www.humanconnectome.org/). There are no relevant accession codes required to access these data, and the authors had no special access privileges that others would not have to the data obtained from any of these databases. Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (https://adni.loni.usc.edu/). The ADNI was launched in 2003 as a public–private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial MRI, positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD).