ARTS: A novel In-vivo classifier of arteriolosclerosis for the older adult brain

Highlights

• •ARTS is an in-vivo classifier of brain arteriolosclerosis.
• •ARTS was trained on ex-vivo brain MRI and neuropathology data.
• •ARTS exhibited good performance in predicting arteriolosclerosis in-vivo.
• •Higher ARTS score was associated with faster two-year decline in cognition.

Abstract

Brain arteriolosclerosis, one of the main pathologies of cerebral small vessel disease, is common in older adults and has been linked to lower cognitive and motor function and higher odds of dementia. In spite of its frequency and associated morbidity, arteriolosclerosis can only be diagnosed at autopsy. Therefore, the purpose of this work was to develop an in-vivo classifier of arteriolosclerosis based on brain MRI. First, an ex-vivo classifier of arteriolosclerosis was developed based on features related to white matter hyperintensities, diffusion anisotropy and demographics by applying machine learning to ex-vivo MRI and pathology data from 119 participants of the Rush Memory and Aging Project (MAP) and Religious Orders Study (ROS), two longitudinal cohort studies of aging that recruit non-demented older adults. The ex-vivo classifier showed good performance in predicting the presence of arteriolosclerosis, with an average area under the receiver operating characteristic curve AUC = 0.78. The ex-vivo classifier was then translated to in-vivo based on available in-vivo and ex-vivo MRI data on the same participants. The in-vivo classifier was named ARTS (short for ARTerioloSclerosis), is fully automated, and provides a score linked to the likelihood a person suffers from arteriolosclerosis. The performance of ARTS in predicting the presence of arteriolosclerosis in-vivo was tested in a separate, 91% dementia-free group of 79 MAP/ROS participants and exhibited an AUC = 0.79 in persons with antemortem intervals shorter than 2.4 years. This level of performance in mostly non-demented older adults is notable considering that arteriolosclerosis can only be diagnosed at autopsy. The scan-rescan reproducibility of the ARTS score was excellent, with an intraclass correlation of 0.99, suggesting that application of ARTS in longitudinal studies may show high sensitivity in detecting small changes. Finally, higher ARTS scores in non-demented older adults were associated with greater decline in cognition two years after baseline MRI, especially in perceptual speed which has been linked to arteriolosclerosis and small vessel disease. This finding was shown in a separate group of 369 non-demented MAP/ROS participants and was validated in 72 non-demented Black participants of the Minority Aging Research Study (MARS) and also in 244 non-demented participants of the Alzheimer's Disease Neuroimaging Initiative 2 and 3. The results of this work suggest that ARTS may have broad implications in the advancement of diagnosis, prevention and treatment of arteriolosclerosis. ARTS is publicly available at https://www.nitrc.org/projects/arts/.

1. Introduction

Brain arteriolosclerosis, one of the main pathologies of cerebral small vessel disease, involves thickening of the vessel wall and stenosis of arterioles (Blevins et al., 2021). Arteriolosclerosis is common in older persons (Arvanitakis et al., 2016) and is more severe in women compared to men (Oveisgharan et al., 2018), and in black compared to white older adults (Barnes et al., 2015). It has been shown that arteriolosclerosis is associated with a lower level of cognitive (Ighodaro et al., 2017, Skrobot et al., 2016, Suemoto et al., 2019) and motor function (Buchman et al., 2013), higher rate of cognitive decline (Kryscio et al., 2016), greater sleep fragmentation (Lim et al., 2016), and higher odds of Alzheimer’s disease (AD) dementia (Arvanitakis et al., 2016). In spite of its frequency and associated morbidity, arteriolosclerosis can only be diagnosed at autopsy.

The purpose of this work was to develop and validate a classifier of arteriolosclerosis based on MRI. To accomplish this, an ex-vivo classifier was first developed using machine learning based on ex-vivo brain MRI and pathology data on participants of the Rush Memory and Aging Project (MAP) and Religious Orders Study (ROS) (Bennett et al., 2018), two longitudinal cohort studies of aging that recruit non-demented older adults. The performance of the ex-vivo classifier in predicting the presence of arteriolosclerosis based on ex-vivo brain MRI data was assessed. The ex-vivo classifier was then translated to in-vivo based on available in-vivo and ex-vivo MRI data on the same participants. The in-vivo classifier was named ARTS (short for ARTerioloSclerosis), is fully automated, and provides a score linked to the likelihood a person suffers from arteriolosclerosis. Tests were conducted to assess the performance of ARTS in predicting the presence of arteriolosclerosis based on in-vivo brain MRI data. Furthermore, the hypothesis that higher ARTS score in non-demented older adults is associated with a greater decline in cognition two years after baseline MRI was tested in MAP/ROS and validated in two independent cohorts.

2. Methods

2.1. Participants

Older adults participating in five studies of aging, MAP, ROS, Clinical Core (CC) of the Rush Alzheimer’s Disease Core Center, Minority Aging Research Study (MARS) (Barnes et al., 2012, Bennett et al., 2018), and Alzheimer's Disease Neuroimaging Initiative (ADNI) (http://adni.loni.usc.edu) were included in this work. MAP, ROS, CC, and MARS are longitudinal, clinical-pathologic studies of aging that enroll community-based older adults without known dementia. CC and MARS enroll older African Americans. ADNI is a public–private partnership with primary goal to test whether serial MRI, positron emission tomography, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (for up-to-date information, see www.adni-info.org). All participants provided written informed consent in accordance with the Declaration of Helsinki and procedures approved by the local institutional committees for the protection of human subjects. All MAP and ROS participants, and a subset of CC and MARS participants also signed an anatomical gift act.

Separate groups of participants were used for training, translation, scan-rescan reproducibility assessment, testing and validation of the classifier. The group used for training the ex-vivo classifier included the 119 MAP/ROS participants that at the time of this work had come to autopsy and had available ex-vivo 3 T MRI and pathology data (Fig. 1). This group will be referred to in the following as the training group (Table 1). The second group was used for translating the ex-vivo classifier to in-vivo and included the 10 MAP/ROS participants from the training group that also had in-vivo 3 T MRI data (Fig. 1). This will be referred to as the translation group (Appendix 1). The third group was used for assessing the scan-rescan reproducibility of the in-vivo classifier (ARTS), included 6 CC participants with two in-vivo MRI scans within the same week, and will be referred to as the reproducibility group (Appendix 1). The fourth group was used for testing the performance of ARTS in predicting the presence of arteriolosclerosis based on in-vivo brain MRI. It included the 79 MAP/ROS participants that at the time of this work had available in-vivo 3 T MRI and pathology data, but no ex-vivo MRI data (Fig. 1). This will be referred to as the testing group (Table 2). There was no overlap between the training and testing groups. The fifth group was used for testing the hypothesis that higher ARTS score in non-demented older adults is associated with a greater decline in cognition two years after baseline in-vivo MRI. It included the 369 MAP/ROS participants that at the time of this work had in-vivo 3 T MRI data while dementia-free, cognitive assessments at the time of MRI and two years later, but no ex-vivo MRI (Fig. 1). This will be referred to as the cognitive decline testing (CDT) group (Table 3). There was no overlap between the training and CDT groups. The sixth and seventh groups were used for validating in two independent cohorts the results of hypothesis testing conducted in the CDT group. More specifically, the sixth group included 72 MARS participants and the seventh group 244 participants of ADNI 2 and 3, that at the time of this work had in-vivo 3 T MRI data while dementia-free and cognitive assessments at the time of MRI and two years later, and will be referred to as the MARS_CDT and ADNI_CDT groups (Table 3). All the above groups included only participants with data that passed quality tests.

Fig. 1.

Venn diagrams describing the available data for the groups of MAP/ROS participants included in training, translating, and testing the classifier. (A) The training group was used for training and testing the ex-vivo classifier. (B) The translation group was used for translating the ex-vivo classifier to in-vivo. (C) The testing group was used for testing the performance of the in-vivo classifier, i.e. ARTS, in predicting the presence of arteriolosclerosis based on in-vivo brain MRI. (D) The cognitive decline testing group (or CDT group), excluding MAP/ROS participants with dementia, was used for testing the hypothesis that higher ARTS score in non-demented older adults is associated with a greater decline in cognition two years after baseline in-vivo MRI. There was no overlap between the training group and the testing or CDT groups.

Table 1.

Demographic, clinical and neuropathologic characteristics of the training group.

Characteristics	No or mild arteriolosclerosis	Moderate or severe arteriolosclerosis	Combined
N (%)	93 (78)	26 (22)	119
Age at death, years (SD)	90.8 (6.2)	92.1 (6.4)	91.1 (6.2)
Male, n (%)	29 (31)	4 (15)	33 (28)
Education, years (SD)	14.9 (3.1)	15.9 (2.9)	15.1 (3.1)
African American, n (%)	1 (1)	0 (0)	1 (0.8)
Median time between last clinical evaluation and death, years	0.6	0.7	0.6
No cognitive impairment at death, n (%)	34 (37)	5 (19)	39 (33)
Mild cognitive impairment at death, n (%)	22 (24)	8 (31)	30 (25)
Dementia at death, n (%)	37 (40)	13 (49)	50 (42)
Mini-mental State Examination (MMSE), mean (SD)	20.6 (9.3)	19.1 (9.2)	20.3 (9.3)
Heart disease, n (%)	17 (18)	6 (23)	23 (19)
Hypertension, n (%)	72 (77)	17 (65)	89 (75)
Diabetes, n (%)	26 (28)	6 (23)	32 (27)
Smoking, n (%)	27 (29)	10 (38)	37 (31)
At least one copy of the ε4 allele, n (%)	17 (18)	8 (31)	25 (21)
Postmortem interval to fixation, hours (SD)	8.1 (2.7)	7.5 (2.3)	8.0 (2.6)
Postmortem interval to 1st ex-vivo MRI, hours (SD)	16.3 (4.9)	16.5 (4.4)	16.3 (4.8)
Postmortem interval to 2nd ex-vivo MRI, days (SD)	31.8 (3.8)	30.8 (2.6)	31.6 (3.6)
High or intermediate NIA Reagan, n (%)	64 (69)	19 (73)	83 (70)
Lewy bodies, n (%)	33 (35)	6 (23)	39 (33)
Hippocampal sclerosis, n (%)	11 (12)	2 (8)	13 (11)
TDP-43 stages 2–3, n (%)	30 (32)	7 (27)	37 (31)
Gross infarcts, n (%)	37 (40)	13 (50)	50 (42)
Microscopic infarcts, n (%)	28 (30)	12 (46)	40 (34)
Moderate or severe atherosclerosis, n (%)	11 (12)	8 (31)	19 (16)
Moderate or severe cerebral amyloid angiopathy, n (%)	32 (34)	7 (27)	39 (33)

Table 2.

Demographic, clinical and neuropathologic characteristics of the testing group.

Characteristics	No or mild arteriolosclerosis	Moderate or severe arteriolosclerosis	Combined
N (%)	63 (80)	16 (20)	79
Age at MRI, years (SD)	87.8 (5.8)	93.4 (4.7)	88.9 (6.0)
Male, n (%)	18 (29)	3 (19)	21 (27)
Education, years (SD)	15.1 (3.1)	16.1 (3.0)	15.3 (3.0)
African American, n (%)	0 (0)	0 (0)	0 (0)
Median antemortem interval between MRI and death, years	2.5	2.1	2.4
No cognitive impairment at MRI, n (%)	39 (62)	8 (50)	47 (59)
Mild cognitive impairment at MRI, n (%)	20 (32)	5 (29)	25 (32)
Dementia at MRI, n (%)	4 (6)	3 (18)	7 (9)
Mini-mental State Examination (MMSE) at MRI, mean (SD)	27.0 (2.6)	27.0 (1.8)	27.0 (2.4)
Heart disease, n (%)	6 (10)	2 (13)	8 (10)
Hypertension, n (%)	38 (60)	14 (88)	52 (66)
Diabetes, n (%)	10 (16)	2 (13)	12 (15)
Smoking, n (%)	26 (41)	4 (25)	30 (38)
At least one copy of the ε4 allele, n (%)	11 (17)	3 (19)	14 (18)
Postmortem interval to fixation, hours (SD)	11.4 (9.6)	9.9 (6.6)	11.1 (9.0)
High or intermediate NIA Reagan, n (%)	41 (65)	11 (69)	52 (66)
Lewy bodies, n (%)	17 (27)	3 (19)	20 (25)
Hippocampal sclerosis, n (%)	3 (5)	2 (13)	5 (6)
TDP-43 stages 2–3, n (%)	17 (27)	7 (44)	24 (30)
Gross infarcts, n (%)	20 (32)	8 (50)	28 (35)
Microscopic infarcts, n (%)	28 (44)	10 (63)	38 (48)
Moderate or severe atherosclerosis, n (%)	11 (17)	5 (31)	16 (20)
Moderate or severe cerebral amyloid angiopathy, n (%)	21 (33)	6 (38)	27 (34)

Table 3.

Demographic and clinical characteristics of the CDT, MARS_CDT, ADNI_CDT groups.

Characteristics	CDT	MARSCDT	ADNICDT
N	369	72	244
Age at MRI, years (SD)	81.2 (7.3)	76.8 (6.0)	76.6 (6.7)
Male, n (%)	78 (21)	12 (17)	131 (54)
Education, years (SD)	15.9 (3.1)	15.5 (3.5)	16.6 (2.5)
African American, n (%)	8 (2)	72 (1 0 0)	11 (5)
No cognitive impairment at MRI, n (%)	311 (84)	68 (94)	134 (55)
Mild cognitive impairment at MRI, n (%)	58 (16)	4 (6)	110 (45)
Mini-mental State Examination (MMSE) at MRI, mean (SD)	28.3 (1.6)	28.6 (1.4)	28.4 (1.8)
Global cognition score at MRI, mean (SD)	0.3 (0.6)	0.2 (0.4)	–
Episodic memory score at MRI, mean (SD)	0.3 (0.7)	0.3 (0.5)	–
Semantic memory score at MRI, mean (SD)	0.4 (0.7)	0.1 (0.6)	–
Working memory score at MRI, mean (SD)	0.2 (0.7)	0.0 (0.7)	–
Perceptual speed score at MRI, mean (SD)	0.3 (0.8)	0.1 (0.8)	–
Visuospatial ability score at MRI, mean (SD)	0.4 (0.7)	−0.1 (0.8)	–
ADAS-Cog at MRI, mean (SD)	–	–	13.5 (5.9)
ADNI-EF at MRI, mean (SD)	–	–	0.8 (0.9)
ADNI-MEM at MRI, mean (SD)	–	–	0.7 (0.7)
Trail Making Test A at MRI, mean (SD)	–	–	33.3 (11.0)
Trail Making Test B at MRI, mean (SD)	–	–	83.6 (40.6)
Heart disease, n (%)	22 (6)	4 (6)	15 (6)
Hypertension, n (%)	216 (59)	59 (82)	112 (46)
Diabetes, n (%)	56 (15)	22 (31)	28 (11)
Smoking, n (%)	161 (44)	32 (44)	43 (18)
At least one copy of the ε4 allele, n (%)	76 (21)	26 (36)	85 (35)

2.2. Cognitive function testing and clinical diagnosis

All participants underwent cognitive function testing at least annually. In this work, performance of MAP, ROS, CC, and MARS participants in five cognitive domains (episodic memory, semantic memory, working memory, perceptual speed, visuospatial ability) was assessed using 19 cognitive tests that were common across the four studies (Marquez et al., 2020). The raw scores from individual tests were first converted to z-scores: $z_i=\frac{c_i-{\stackrel{-}{c}}_i}{s_i}$ , where $z_i$ and $c_i$ are a participant’s z-score and raw score for test i, and ${\stackrel{-}{c}}_i$, $s_i$ are the mean and standard deviation of the raw scores across participants, and the resulting z-scores were averaged into a composite score for each cognitive domain, and also averaged across all tests to obtain a composite score of global cognition (Wilson et al., 2015): $\mathit{composite}=\frac{{\sum }_{i=j}^k{z}_i}{n}$, where j,k are indices of the first and last tests included in a composite, and n is the number of tests included in that composite. The cognitive measures considered in this work for ADNI2/3 participants included the score for the 13-item Alzheimer's Disease Assessment Scale–cognitive subscale (Mohs et al., 1997) (ADAS-Cog), the composite scores for executive function (Gibbons et al., 2012) (ADNI-EF) and memory (Crane et al., 2012) (ADNI-MEM), and the scores for the Trail Making Test Part A (TMT-A) and Part B (TMT-B). The latter two tests were selected because they assess executive abilities that are similar to those captured in the perceptual speed domain of MAP, ROS, CC, and MARS. The Mini-Mental State Examination (MMSE) was used for descriptive purposes in all participants.

Clinical diagnosis of dementia followed accepted and validated criteria (McKhann et al., 1984). Participants were classified as having MCI if they had cognitive impairment but did not meet the criteria for dementia (Bennett et al., 2002, Boyle et al., 2006). Persons without dementia or MCI were categorized as no cognitive impairment (NCI). For the MAP/ROS participants that came to autopsy, all available clinical data were reviewed by a neurologist and a summary final diagnostic opinion was provided blinded to all postmortem data.

2.3. In-vivo MRI data collection and processing

In-vivo MRI data were collected for the translation, testing and CDT groups on a single 3 T Siemens MRI scanner, for the reproducibility and MARS_CDT groups on a single 3 T Philips scanner, and for the ADNI_CDT group on several 3 T General Electric, Philips and Siemens scanners. High resolution T1-weighted, T2-weighted fluid-attenuated inversion recovery (FLAIR), and diffusion-weighted MRI data were collected on all participants. The protocols used on the different scanners are summarized in Appendix 2.

Bias field inhomogeneity was corrected in raw T1-weighted and T2-weighted FLAIR images (Tustison et al., 2010). Gray and white matter were segmented based on T1-weighted images using CAT12 (Farokhian et al., 2017). White matter hyperintensities (WMH) were segmented using a deep learning model based on T2-weighted FLAIR and T1-weighted images (Li et al., 2018), and the total WMH volume in the cerebrum was normalized by the total cerebral white matter volume. The diffusion-weighted data were corrected (diffprep, TORTOISE, http://www.tortoisedti.org) for distortions due to eddy-currents as well as for bulk motion by registration to the b = 0 sec/mm² volume, the B-matrix was reoriented, diffusion tensors were calculated (diffcalc, TORTOISE), and maps of fractional anisotropy (FA) were generated (Pierpaoli et al., 2010).

2.4. Brain hemisphere preparation and ex-vivo MRI data collection

When MAP/ROS participants from the training, translation, and testing groups came to autopsy, a technician removed the brain, and divided the cerebrum into left and right hemispheres. The hemisphere with more visible pathology was immersed within 30 min of removal from the skull in phosphate-buffered 4% formaldehyde solution with its medial aspect facing the bottom of the container and was imaged with diffusion-weighted MRI (only for the training and translation groups) within 24 h postmortem (ex-vivo MRI protocols are summarized in Appendix 3). After ex-vivo MRI, the hemisphere was refrigerated at 4 °C. At approximately 30 days postmortem the refrigerated hemisphere was allowed to return to room temperature and was imaged with MRI a second time using a multi-echo spin-echo sequence (Appendix 3). Within 2 weeks after the second ex-vivo MRI session, gross examination and histopathologic diagnostic examination were performed by a board-certified neuropathologist.

2.5. Ex-vivo MRI data processing

Bias field inhomogeneity was corrected in raw ex-vivo multi-echo spin-echo data (Tustison et al., 2010). Gray and white matter and WMH were segmented in ex-vivo multi-echo spin-echo data using in-house software combining SPM12 (Ashburner and Friston, 2005) and FSL (Zhang et al., 2001) tools and a custom brain template. The total WMH volume was normalized by the total white matter volume. The ex-vivo diffusion-weighted data were corrected for distortions due to eddy-currents by registration to the b = 0 sec/mm² volume, the B-matrix was reoriented, diffusion tensors were calculated, and maps of fractional anisotropy (FA) were generated using the same methods as those used in-vivo (Pierpaoli et al., 2010).

2.6. Neuropathologic evaluation

Following the second ex-vivo MRI scan, each hemisphere was sectioned into 1 cm thick coronal slabs. The slabs were evaluated macroscopically. Selected tissue blocks were dissected, embedded in paraffin, cut into sections, and mounted on glass slides. A board-certified neuropathologist blinded to all clinical and ex-vivo MRI data completed neuropathologic evaluation following well-established procedures. Arteriolosclerosis was assessed on hematoxylin and eosin stained sections of the anterior basal ganglia based on the degree of concentric hyaline thickening of the wall of small arterioles (<50 μm) (Arvanitakis et al., 2017).

Arteriolosclerosis severity ranged from 0 if there was no arteriolosclerosis, 1 (mild) if arteriole walls were minimally thickened, 3 (moderate) if arteriolar wall thickness was increased up to 2 times the normal thickness, 5 (severe) if arteriolar wall thickness was higher than two times the normal thickness. Arteriolosclerosis severity was then compressed into four levels: none, mild, moderate, and severe. Atherosclerosis was assessed based on the number and extent of vascular involvement at the circle of Willis and was rated as none, mild, moderate, or severe. Cerebral amyloid angiopathy was assessed in 4 neocortical regions and was rated as none, mild, moderate, or severe (Arvanitakis et al., 2017). Gross infarcts of any age were scored as none, one, or more than one. Microscopic infarcts of any age were detected in a minimum of nine regions and were also scored as none, one, or more than one (Arvanitakis et al., 2017). Amyloid-β plaques and neurofibrillary tangles were assessed in 8 brain regions, and separate composite measures were constructed for β-amyloid burden and tangle density (Oveisgharan et al., 2018). Transactive Response DNA-binding Protein 43 (TDP-43) pathology was rated on four levels (Nag et al., 2015). Hippocampal sclerosis was rated as present or absent (Nag et al., 2015). Lewy bodies were detected in seven regions and were rated as present or absent (Schneider et al., 2012).

2.7. Development of an ex-vivo classifier of arteriolosclerosis

A logistic regression classifier was trained to distinguish participants with a pathological diagnosis of moderate or severe arteriolosclerosis from those with none or mild arteriolosclerosis, based on features extracted from ex-vivo MRI as well as basic demographic information. One ex-vivo MRI-derived feature considered was the normalized WMH volume. Regional ex-vivo FA values were also considered as features. The regions of interest (ROI) were defined by first projecting (Smith et al., 2006) FA values from participants of the training group onto the white matter skeleton of the IIT Human Brain Atlas v.5.0. (Zhang and Arfanakis, 2018) and conducting voxel-wise linear regression to determine clusters of voxels showing negative association between FA and arteriolosclerosis, controlling for all other neuropathologies, age at death, sex, years of education, scanner, postmortem interval to immersion in fixative solution (PMI), and postmortem interval to ex-vivo MRI (PMIi). Four ROIs showing the strongest negative association of FA with arteriolosclerosis were determined (Fig. 2), and the median ex-vivo FA values from these ROIs were considered as separate features. To control for the potentially confounding effects of using different scanners, different PMI and PMIi, the five ex-vivo MRI-based features were corrected for scanner, PMI and PMIi using linear regression models (here the WMH feature was log-transformed to account for its skewed distribution). In addition to the five corrected ex-vivo MRI-based features, age and sex were also considered as features in the classifier.

Fig. 2.

Median FA values were extracted from the four numbered regions and were used as features in the classifier. The regions are displayed in the space of the IIT Human Brain Atlas v.5. The mean FA template and white matter skeleton of the atlas are shown in grayscale and green color, respectively. Note that the regions are only defined on the skeleton and have been dilated in this figure to enhance visualization. According to the Regionconnect software (Qi and Arfanakis, 2021) the top five most likely connections through each of the four regions are: (Region 1) lateral occipital to middle temporal (13.5%), fusiform to lateral occipital (13.3%), inferior temporal to lateral occipital (10.1%), middle temporal to pericalcarine (9%), fusiform to lingual (6.2%); (Region 2) lateral occipital to superior temporal (9.9%), middle temporal to superior temporal (6.6%), superior parietal to superior temporal (6.2%), inferior parietal to superior temporal (6%), banks of the superior temporal sulcus to superior temporal (5.5%); (Region 3) precentral to thalamus (17.8%), paracentral to thalamus (10.7%), superior frontal to superior parietal (9.6%), precentral to caudate (6.1%), postcentral to precentral (4.3%); (Region 4) precuneus to superior parietal (12%), inferior parietal to superior parietal (6.7%), left superior parietal to right superior parietal (6.5%), isthmus cingulate to superior parietal (6.5%), cuneus to superior parietal (6.1%). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

The performance of an ex-vivo classifier based on the above features was compared to that of classifiers based on: a) demographic features, b) WMH and demographic features, or c) regional FA and demographic features. To assess performance in each case, stratified shuffle split cross-validation was repeated 100 times, each time using 80% of the data for training and 20% for testing, and the average area under the receiver operating characteristic curve (AUC) was computed. The regularization strength of the logistic regression was tuned using 10-fold cross-validation in the training set of each repeat. The two-sided Wilcoxon signed rank test was used to compare the performance of the classifier with the highest average AUC to that of others (p < 0.05). Bonferroni correction was applied. The final classifier was trained based on the set of features resulting in highest performance and 100% of the data from the training group.

2.8. Development of the ARTS software for in-vivo classification of arteriolosclerosis

To translate the ex-vivo classifier to in-vivo, the relationship between ex-vivo and in-vivo values of the MRI-based features was established based on data from the translation group. More specifically, the relationship between ex-vivo and in-vivo values of the WMH feature was established using simple linear regression (log-transformed WMH values were used in this step), and that of the regional FA features was established using a linear mixed effects model with random effects on the slope. The in-vivo classifier of arteriolosclerosis was packaged into a fully automated software container named ARTS. This is a software container that takes as input raw MRI data and demographics, performs automatically all the necessary image processing described in Section 2.3, extracts the necessary features, runs the in-vivo classifier, and provides a score linked to the likelihood a person suffers from arteriolosclerosis. ARTS was made publicly available at: https://www.nitrc.org/projects/arts/.

2.9. In-vivo assessment of ARTS scan-rescan reproducibility

To assess the scan-rescan reproducibility of ARTS, the participants of the reproducibility group were imaged with MRI twice within one week and the ARTS score was generated for both scans. The intraclass correlation (ICC) was calculated as a measure of ARTS reproducibility.

2.10. In-vivo assessment of ARTS performance: prediction of pathology

Tests were conducted to assess the performance of ARTS in predicting in-vivo the presence of arteriolosclerosis. For that purpose, ARTS was applied on the in-vivo data of the testing group and the AUC for predicting a pathological diagnosis of moderate or severe arteriolosclerosis was calculated. Since the chance of additional pathology developing after in-vivo MRI increases with longer intervals between in and vivo MRI and death, which can distort estimates of the performance of ARTS in predicting pathology, the AUC was also calculated for subgroups of participants of the testing group with different ranges of antemortem intervals (AMI). In addition, linear regression was used to investigate the association of in-vivo ARTS score (dependent variable) with arteriolosclerosis severity assessed pathologically (independent variable) controlling for AMI and the interaction of severity assessed pathologically with AMI.

2.11. In-vivo assessment of ARTS performance: association with cognitive decline

The hypothesis that a higher ARTS score in non-demented older adults is associated with a greater decline in cognition two years after baseline MRI was tested in the CDT group, and validated in the MARS_CDT, and ADNI_CDT groups. The short interval of two years was selected because it can be reproduced in clinical trials and other interventional studies. First, ARTS was applied on the in-vivo data of the CDT group and linear regression was used to investigate the association of two-year change in global cognition (dependent variable) with in-vivo ARTS score (independent variable) controlling for years of education. The same analysis was repeated using two-year change in episodic memory, semantic memory, working memory, perceptual speed, and visuospatial abilities. Second, since the CDT group included participants from the same studies (MAP/ROS) as those used for training the classifier (though no overlap existed between the CDT and training groups) the same hypothesis was tested in both the MARS_CDT and ADNI_CDT groups, which included participants of different studies (MARS and ADNI2/3, respectively). Due to differences in cognitive batteries across studies, the analysis conducted in ADNI_CDT participants involved two-year change in ADAS-Cog, ADNI-EF, ADNI-MEM, TMT-A and TMT-B.

2.12. Data and software availability

The data used in this work can be assessed by submitting a request to www.radc.rush.edu. ARTS is publicly available at: https://www.nitrc.org/projects/arts/.

3. Results

3.1. Testing the ex-vivo classifier of arteriolosclerosis

The average AUC for ex-vivo classification of arteriolosclerosis in the training group (Table 1) based on WMH, regional FA and demographic features was 0.78 ± 0.10 (Fig. 3). This was higher (two-sided Wilcoxon signed rank test, p < 0.0001) than the average AUC when using regional FA and demographic features (0.73 ± 0.12), or WMH and demographic features (0.72 ± 0.11), or only demographic features (0.59 ± 0.13). Therefore, the final ex-vivo classifier was trained based on WMH, regional FA and demographic features, with WMH and regional FA having higher contribution than demographics in classification of arteriolosclerosis.

Fig. 3.

ROC curves for ex-vivo classification of arteriolosclerosis using classifiers based on different combinations of features. The black line and gray cloud represent the mean and standard deviation of the best performing classifier that combines WMH, regional FA, and demographic features.

3.2. In-vivo assessment of ARTS performance: Prediction of pathology

After translating the ex-vivo classifier to in-vivo and packaging it into the fully automated ARTS software, it was shown that the intraclass correlation for the in-vivo scan-rescan reproducibility of ARTS score was excellent ICC = 0.99. The performance of ARTS in predicting in-vivo the presence of arteriolosclerosis was assessed in the testing group (Table 2). Compared to the training group, the testing group exhibited a much lower frequency of dementia (9% compared to 42% of the training group), higher MMSE by 7 points (two-sided Wilcoxon rank-sum test, p < 0.0001), lower age by 2 years (two-sided Student’s t-test, p = 0.015), but similar neuropathological burden (other than microscopic infarcts which affected 48% of the testing group and 34% of the training group). The AUC for in-vivo classification of arteriolosclerosis in the testing group is shown in Fig. 4 for subgroups of participants with different ranges of antemortem intervals (AMI). For participants with AMI ≤ 2.4 years AUC = 0.79, while for those with AMI ≤ 3.6 years AUC = 0.69, and for those with AMI ≤ 6.8 years AUC = 0.68. Furthermore, linear regression in the whole testing group showed that the in-vivo ARTS confidence score was associated with arteriolosclerosis severity assessed pathologically (p = 0.001), controlling for AMI and its interaction with severity assessed pathologically (Fig. 5).

Fig. 4.

AUC values for in-vivo prediction of arteriolosclerosis in participants of the testing group with antemortem intervals (AMI) less than the threshold value shown on the horizontal axis. The number of participants with AMI less than the threshold is indicated above the curve.

Fig. 5.

In-vivo ARTS confidence scores at different levels of arteriolosclerosis severity assessed pathologically for participants of the testing group with AMI ≤ 2.4 years. Stages 2 and 3 were pooled because stage 3 included only one participant. Boxes represent ARTS scores from 25th-75th percentiles, the horizontal lines inside the boxes represent median scores, and the whiskers illustrate the range of variation.

3.3. In-vivo assessment of ARTS performance: Association with cognitive decline

The hypothesis that a higher ARTS score in non-demented older adults is associated with a greater decline in cognition two years after baseline MRI was tested in the CDT group, and validated in the MARS_CDT and ADNI_CDT groups. First, compared to the training group, CDT participants were free of dementia, had 8 more points on MMSE (two-sided Wilcoxon rank-sum test, p < 0.0001), were 10 years younger (two-sided Student’s t-test, p < 0.0001), had 1 year more education (two-sided Student’s t-test, p = 0.017), and lower frequency of vascular risk factors (except for history of smoking) (Table 3). A higher ARTS score in the CDT group was associated with greater two-year decline in global cognition (model estimate = −0.23, p = 0.000017), perceptual speed (model estimate = −0.18, p = 0.00084), semantic memory (model estimate = −0.18, p = 0.00075), episodic memory (model estimate = −0.18, p = 0.00067), and visuospatial abilities (model estimate = −0.12, p = 0.028).

The MARS_CDT participants were African American and free of dementia, were 8 points higher on MMSE (two-sided Wilcoxon rank-sum test, p < 0.0001), were 14 years younger (two-sided Student’s t-test, p < 0.0001), had lower frequency of heart disease but higher frequency of smoking, and higher frequency of at least one copy of the APOE ε4 allele compared to the training group (Table 3). A higher ARTS score in the MARS_CDT group was associated with greater two-year decline in perceptual speed (model estimate = −0.29, p = 0.016).

ADNI_CDT participants were free of dementia, had 8 points higher MMSE (two-sided Wilcoxon rank-sum test, p < 0.0001), were 15 years younger (two-sided Student’s t-test, p < 0.0001), had twice the percentage of males, had 1.5 years more education (two-sided Student’s t-test, p < 0.0001), lower frequency of vascular risk factors, and higher frequency of at least one copy of the APOE ε4 allele compared to the training group. In addition, clinical diagnosis of ADNI_CDT participants was more evenly distributed between NCI and MCI compared to that in CDT and MARS_CDT participants. A higher ARTS score in the ADNI_CDT group was associated with greater two-year increase in ADAS-Cog (model estimate = 0.14, p = 0.033), TMT-A (model estimate = 0.13, p = 0.039) and TMT-B (model estimate = 0.19, p = 0.0026), and greater two-year decline in ADNI-MEM (model estimate = -0.14, p = 0.039).

Overall, after Bonferroni correction for multiple comparisons, it was shown that a higher ARTS score in the CDT group was associated with greater two-year decline in global cognition, perceptual speed, episodic and semantic memory, and in the ADNI_CDT group with greater two-year increase in TMT-B.

4. Discussion

A novel MRI-based in-vivo classifier of brain arteriolosclerosis named ARTS was developed, tested and validated in this work. ARTS was trained on ex-vivo brain MRI and neuropathology data from community-based older adults, was translated to in-vivo and was made fully automated and publicly available at: https://www.nitrc.org/projects/arts/. ARTS is based on features related to WMH, regional FA and demographics, with WMH and regional FA having higher contribution than demographics in classification of arteriolosclerosis. ARTS exhibited good performance in predicting in-vivo the presence of arteriolosclerosis in a group that was largely dementia-free, and achieved excellent scan-rescan reproducibility. Higher ARTS scores in non-demented older adults were associated with faster two-year decline in cognition, especially in cognitive abilities that have been traditionally linked to arteriolosclerosis and small vessel disease. This finding was validated in two independent cohorts. The results of this work suggest that ARTS may have broad application in the advancement of clinical diagnosis, prevention and treatment of arteriolosclerosis.

The performance of ARTS in predicting the presence of arteriolosclerosis in-vivo was good, with an AUC approaching 0.80. This level of performance in-vivo is notable considering the fact that arteriolosclerosis can only be diagnosed at autopsy. In addition, the fact that the testing group was almost exclusively free of dementia and more than half of the participants had no cognitive impairment indicates that ARTS may allow early in-vivo prediction of arteriolosclerosis. Based on the above, ARTS may have important clinical implications. First, it may allow refined participant selection in clinical drug and prevention trials. Second, when treatments become available, ARTS may be used to predict pathology prior to the onset of substantial cognitive deficits, when treatments are expected to have the best clinical outcome. Furthermore, since ARTS is, to our knowledge, the first in-vivo classifier of arteriolosclerosis, it sets the baseline for further development of in-vivo predictors of this important neuropathology.

A higher ARTS score was associated with greater two-year decline in cognition in non-demented older adults, as shown in one group (CDT) and validated in two independent cohorts (MARS and ADNI). After Bonferroni correction for multiple comparisons, a higher ARTS score was associated with faster two-year decline in global cognition, perceptual speed, episodic and semantic memory in the CDT group, and with two-year increase in TMT-B in the ADNI_CDT group. In MARS_CDT, a higher ARTS score was also associated with greater two-year decline in perceptual speed, but this finding did not survive Bonferroni correction, probably because of the small size of the MARS_CDT group (5.1 and 3.4 times smaller than the CDT and ADNI_CDT groups, respectively). The above findings are particularly interesting since perceptual speed and TMT-B assess similar cognitive abilities that are known to be affected by arteriolosclerosis and small vessel disease (Arvanitakis et al., 2016, Cosentino et al., 2011, Debette and Markus, 2010, Duering et al., 2014, MacPherson et al., 2017), indicating that ARTS targets the expected cognitive domain and does so consistently across cohorts. Also, these findings suggest that ARTS may be used to identify non-demented individuals at higher risk of cognitive decline and therefore in higher need of treatment.

The approach of combining ex-vivo instead of in-vivo MRI and pathology data to train ARTS had important advantages without limiting the suitability of ARTS for in-vivo application. First, the use of ex-vivo instead of in-vivo MRI eliminated the possibility of additional pathology developing after MRI data collection, ensuring that MRI and pathology data correspond to the same condition of the brain. In contrast, when combining in-vivo MRI and pathology data, long antemortem intervals a) may cause MRI to image a healthier state of the brain than that at autopsy leading to an underestimation of the ability of MRI to detect pathology-related brain abnormality, and b) may increase the variation in MRI findings across persons with the same neuropathological burden. Consequently, in-vivo MRI with long antemortem intervals may negatively impact the training and testing of classifiers of pathology. The negative impact on testing was demonstrated in this work in the form of lower AUC values in older adults with antemortem intervals longer than 2.4 years. A second advantage of the use of ex-vivo MRI is that it allows imaging independent of frailty level, which, in addition to using data from community-based older adults, enhanced generalizability of ARTS. At the same time, the use of ex-vivo MRI data for training did not limit the suitability of ARTS for in-vivo application. Although the testing, CDT, MARS_CDT, and ADNI_CDT groups were imaged in-vivo and were cognitively healthier and younger than the training group, the AUC for predicting arteriolosclerosis in-vivo was similar to that ex-vivo, and the ARTS score showed a meaningful association with two-year decline in cognition.

ARTS exhibited excellent scan-rescan reproducibility. Therefore, application of ARTS in longitudinal studies would allow high sensitivity in detecting small changes. This will be important for studying individual trajectories and monitoring response to potential treatments.

The current study has multiple major strengths and also a few limitations. The strengths include a) training ARTS based on detailed pathology data, b) using ex-vivo MRI to eliminate the ante-mortem interval in the training data, c) including testing and validation groups that have no overlap with the training group, d) using two external validation groups derived from studies other than the one used for training and with different recruitment and imaging protocols, e) testing ARTS in large numbers of non-demented older adults, f) using both pathologic and cognitive assessments to investigate the performance of ARTS, g) developing fully automated software for ARTS, h) assessing its scan-rescan reproducibility, and i) making it publicly available to support open science. One limitation of the current study is the relatively small number of persons in the training (N = 119) and testing (N = 79) groups due to the limited availability of 3 T MRI and pathology data on the same older adults, especially in persons free of dementia. Ongoing investigations on hundreds of non-demented MAP/ROS/MARS participants that have agreed to longitudinal MRI and have signed an anatomical gift act will address this limitation in the coming years. As new data become available, ARTS will be updated and the new versions will be shared. Another limitation is the relatively small number of persons in one of the validation groups (MARS_CDT). This was mitigated by including a second validation group (ADNI_CDT) with 3.4 times more participants. Additional testing in racial and ethnic minorities is necessary. The youngest group ARTS was tested on was 76.6 ± 6.7 years of age and therefore additional testing in younger individuals is warranted. Further testing of the association of ARTS with change in cognition is also necessary to control for additional factors known to be correlated with cognition and cognitive decline.

In conclusion, this work developed, tested and validated a novel, fully automated MRI-based in-vivo classifier of arteriolosclerosis named ARTS. ARTS exhibited good performance in predicting in-vivo the presence of arteriolosclerosis in a group that was largely dementia-free, and its score in non-demented older adults was associated with faster two-year decline in cognition, especially in cognitive abilities that have been linked to arteriolosclerosis and small vessel disease. Based on these findings, ARTS may have broad application in the advancement of clinical diagnosis, prevention and treatment of arteriolosclerosis.

Funding

The study was supported by NIH grants P30AG010161, UH2NS100599, UH3NS100599, R01AG064233, R01AG15819, RF1AG022018, R01AG056405, R01AG17917, R01AG067482, and the Illinois Department of Public Health (Alzheimer’s Disease Research Fund).

In addition, part of the data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12–2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and generous contributions from the following: AbbVie; Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org/). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

6. Data and Software Availability Statement

The data used in this work can be assessed by submitting a request to www.radc.rush.edu. ARTS is publicly available at https://www.nitrc.org/projects/arts/.

CRediT authorship contribution statement

Nazanin Makkinejad: Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing - original draft, Writing - review & editing. Arnold M. Evia: Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing - review & editing. Ashish A. Tamhane: Data curation, Methodology, Writing - review & editing. Carles Javierre-Petit: Methodology, Writing - review & editing. Sue E. Leurgans: Formal analysis, Writing - review & editing. Melissa Lamar: Methodology, Writing - review & editing. Lisa L. Barnes: Resources, Funding acquisition, Writing - review & editing. David A. Bennett: Resources, Funding acquisition, Writing - review & editing. Julie A. Schneider: Conceptualization, Funding acquisition, Investigation, Project administration, Resources, Writing - review & editing. Konstantinos Arfanakis: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: