Plasma Biomarkers as Predictors of Progression to Dementia in Individuals with Mild Cognitive Impairment

Abstract

Background:

Blood-based biomarkers (BBMs) are of growing interest in the field of Alzheimer’s disease (AD) and related dementias.

Objective:

This study aimed to assess the ability of plasma biomarkers to 1) predict disease progression from mild cognitive impairment (MCI) to dementia and 2) improve the predictive ability of magnetic resonance imaging (MRI) and cerebrospinal fluid (CSF) measures when combined.

Methods:

We used data from the Alzheimer’s Disease Neuroimaging Initiative. Machine learning models were trained using the data from participants who remained cognitively stable (CN-s) and with Dementia diagnosis at 2-year follow-up visit. The models were used to predict progression to dementia in MCI individuals. We assessed the performance of models with plasma biomarkers against those with CSF and MRI measures, and also in combination with them.

Results:

Our models with plasma biomarkers classified CN-s individuals from AD with an AUC of 0.75 ± 0.03 and could predict conversion to dementia in MCI individuals with an AUC of 0.64 ± 0.03 (17.1% BP, base prevalence). Models with plasma biomarkers performed better when combined with CSF and MRI measures (CN versus AD: AUC of 0.89 ± 0.02; MCI-to-AD: AUC of 0.76 ± 0.03, 21.5% BP).

Conclusions:

Our results highlight the potential of plasma biomarkers in predicting conversion to dementia in MCI individuals. While plasma biomarkers could improve the predictive ability of CSF and MRI measures when combined, they also show the potential to predict non-progression to AD when considered alone. The predictive ability of plasma biomarkers is crucially linked to reducing the costly and effortful collection of CSF and MRI measures.

INTRODUCTION

Alzheimer’s disease (AD) is one of the most common neurodegenerative diseases, currently affecting more than 6.5 million Americans aged 65 and older, projected to more than double by 2060 [1]. Pathologic brain changes of AD typically precede clinical symptoms [2]. In absence of disease modifying therapies, it is expected that individuals with pathologic changes specific to AD progress to dementia at varying rates [3]. Individuals with mild cognitive impairment (MCI) are at much higher risk of progression to dementia [4], with annual conversion rate from Mayo defined MCI to dementia ranging between 5 to 10% [3]. The ability to predict when and if individuals with MCI will progress to dementia has great implications for tailoring preventive measures, developing disease modifying therapies, and monitoring disease progression and response to treatments in individuals with AD brain pathology.

Hallmark pathological brain changes in AD include slow aggregation of amyloid-β (Aβ) protein, accumulation of hyperphosphorylated tau (p-tau) and a parallel functional and structural neuronal degradation [5, 6]. While postmortem pathology remains the gold-standard for measurement of AD pathology, imaging and biofluid biomarkers provide in vivo estimates of brain pathology with varying degree of accuracy [7]. Imaging biomarkers including structural magnetic resonance imaging (MRI) and positron emission tomography (PET) scans (with radiolabeled tracers specific to a protein) have been shown to be effective tools for early diagnosis of AD [8, 9]. On the other hand, cerebrospinal fluid (CSF) measures provide markers for Aβ deposition and tau accumulation [10], which together provide a high predictive value for disease progression in MCI individuals [11]. Furthermore, recent studies have confirmed diagnostic and prognostic utility of Alzheimer’s disease and related dementias blood-based biomarkers (ADRD-BBMs) [12]. Results from several cohorts show high potential for implementation of the core pathological biomarkers (i.e., Aβ and phosphorylated tau, p-tau) [13, 14] and of blood-based biomarkers (BBMs) of neurodegeneration (e.g., neurofilament light chain, NfL) [13, 15].

Several studies have shown that various plasma biomarkers can predict cognitive decline in the cognitively unimpaired [13, 16] and predict neuropathologic changes in AD [17]. In addition, a variety of proteomic biomarkers and AD-pathology related biomarkers in plasma have also performed well in distinguishing diagnostic groups (AD-Dementia and cognitively normal) [18, 19] and in predicting progression from MCI to Alzheimer’s Dementia [19, 20]. Previous studies have investigated the joint and separate value of CSF and imaging biomarkers to predict MCI-to-AD conversion [21, 22]. However, few studies have investigated the utility of plasma biomarkers specific to the ATN classification, also measurable in other biofluids such as CSF, in predicting the conversion of MCI-to-AD. In this study, we aimed to establish whether BBMs, provided by measures from plasma assays, can provide performance comparable to the same pathological biomarkers from more invasive CSF, in terms of diagnosis and prediction of disease progression in AD. As the field of clinical practice and research in AD starts to identify the value of validated BBMs at low cost, it is crucial for the trials and studies to rightly estimate the necessity and value-addition of employing each of the blood tests, CSF and imaging measures for screening, depending on the goals of the study. Therefore, we studied the differences between predictive abilities of plasma biomarkers and those from CSF and MRI, both individually and together, to predict the progression from MCI to AD dementia. First, we studied the ability of plasma biomarkers to predict clinical progression from MCI to dementia (MCI-conversion). Further, we studied the ability of different combinations of biomarkers (i.e., MRI, CSF, and BBMs) and the contribution of each individual measure to the models to predict MCI-conversion by assessing BBM, CSF and MRI measures individually, addition of BBMs to CSF measures, and the addition of BBMs to the combination of CSF and MRI. Finally, since models integrating neuropsychological assessments have demonstrated great performance in accurately predicting the progression [23, 24], we compared predictive value of conventional neuropsychological tests with biomarkers in this population.

MATERIALS AND METHODS

ADNI study design

We used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) downloaded from the Laboratory of Neuro Imaging (LONI) website (http://www.adni.loni.usc.edu) in January 2022. The ADNI study was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. ADNI is an ongoing cohort with the phases ADNI-1, ADNI-GO, ADNI-2, and ADNI-3 across several participating institutions. ADNI was approved by the institutional review boards at all the participating institutions. Written informed consent was obtained by or on behalf of all the participants at each site. The ADNI database provides data from several modalities such as clinical and neuropsychological assessments, serial MRI, amyloid PET, biomarkers from CSF and blood plasma. The primary goal of ADNI has been to test whether these markers can be combined to measure the progression of MCI and early AD. For up-to-date information, see http://www.adni-info.org.

Participants

Inclusion criteria in this study were having at least one wave of follow-up and confirmed clinical diagnosis at baseline and follow up. Figure 1 provides a flowchart of study participants. A total of 2013 participants met inclusion criteria for this study. The Cognitively Normal (CN), MCI (early and late), and Dementia groups comprised of 700, 963, and 350 respectively at the baseline visit. Table 1 summarizes characteristics of all the participants included in this study. Participants were categorized into one of the following subgroups based on follow up data:

• Stable CN (CN-s, N = 534): CN individuals who remained CN for at least 2 years of follow up

• Progressive CN (CN-p, N = 35): CN individuals at baseline who progressed to MCI or Dementia within the first 2 years of follow up.

• Stable MCI (MCI-s, N = 541): Amnestic MCI individuals at baseline who did not progress to dementia during 2 years of follow up.

• Progressive MCI (MCI-p, N = 193): Amnestic MCI individuals at baseline who progressed to dementia during 2 years of follow up.

• Dementia due to AD: (AD, N = 131): Participants with clinical diagnosis of dementia and biomarker confirmation of AD status (Amyloid-β positive based on PET)

Fig. 1. Overview of the participants included in the current study.

CN, Cognitively Normal; CN-s, Stable CN; MCI, Mild Cognitive Impairment; AD, Alzheimer’s Disease; Aβ+, amyloid-β positive; *5 individuals were ignored in this study whose baseline diagnosis Dementia reverted in future visits.

Table 1.

Overview of the participants according to their baseline diagnosis

		All Participants	CN	MCI	Dementia	p
	Count (%)	2013	700 (35)	960 (48)	350 (17)
Demographics	Age (y), mean (SD)	73.4 (7.1)	73.2 (6.1)	73.0 (7.5)	74.7 (7.8)	0.001
	Education (y), mean (SD)	16.0 (2.8)	16.5 (2.6)	16.0 (2.8)	15.3 (2.9)	<0.001
	Female, N (%)	925 (45)	382 (54)	390 (40)	153 (43)	<0.001
	Non-Hispanic White, N (%)	1,812 (90)	617 (88)	878 (91)	317 (90)	0.160
	Married, N (%)	1,548 (76)	495 (70)	757 (78)	296 (84)	<0.001
APOE ε4 alleles N (%)	0	1,064 (52)	485 (69)	470 (48)	109 (31)	<0.001
	1	730 (36)	192 (27)	372 (38)	166 (47)
	2	194 (9)	19 (2)	104 (10)	71 (20)
Neuropsych.	RAVLT Immediate	36.2 (12.7)	45.5 (9.9)	34.3 (10.6)	22.8 (7.2)	<0.001
Tests mean (SD)	RAVLT Learning	4.4 (2.8)	6.1 (2.3)	4.1 (2.6)	1.8 (1.8)	<0.001
	RAVLT Forgetting	4.3 (2.6)	3.7 (3.0)	4.6 (2.5)	4.5 (1.8)	<0.001
	ADAS (cog, 11)	10.3 (6.5)	5.7 (2.9)	10.2 (4.5)	19.4 (6.6)	<0.001
	TMT-B	116.8 (73.4)	82.2 (42.0)	115.9 (66.0)	195.5 (87.5)	<0.001

p, One-way ANOVA (for continuous variables) and chi-square test of independence (for categorical variables) between the diagnostic groups w.r.t each of the participant demographic and neuropsychological characteristic; RAVLT, Rey’s Auditory Verbal Learning Test, individually considering three of its components – immediate, learning, forgetting; ADAS-Cog, Alzheimer’s Disease Assessment Scale–Cognitive Subscale (11-item subscore); TMT-B, Trail Making Test – Part B.

Neuropsychological assessments

At each visit, participants were assigned to one of the 3 diagnostic groups: cognitively normal (CN), amnestic mild cognitive impairment (MCI), and dementia. CN participants had Mini-Mental State Examination (MMSE) scores of 24 or higher and a Clinical Dementia Rating (CDR) score of 0. All MCI participants were diagnosed as amnestic; this diagnostic classification required MMSE scores between 24 and 30 (inclusive), a memory complaint, objective memory loss measured by education adjusted scores on the Wechsler Memory Scale Logical Memory II, a CDR of 0.5, absence of significant impairment in other cognitive domains, essentially preserved activities of daily living, and absence of dementia. Since ADNI was designed as a simulated randomized controlled trial, the specific implementation of the criteria for MCI had to be as unambiguous as possible to allow maximal reliability across participating sites [25]. All CN participants selected for this study remainedcognitivelynormalwithinthefirst6months of follow-up during the first year of follow-up. The subjects with dementia had to satisfy the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) criteria for clinically defined probable AD, have MMSE scores between 20 and 26 (inclusive), and CDR of 0.5 or 1.

Of the standard cognitive measurements that were available through ADNI study, we focused on a few of them that are known to assess working memory and other common AD-related memory processes: Rey’s Auditory Verbal Learning Test (RAVLT), individually considering three of its components – immediate, learning, forgetting [26]; Alzheimer’s Disease Assessment Scale-Cognitive Subscale, the 11-item sub-score (ADAS-Cog) [27]; Trail Making Test – Part B (TMT-B) [28]. We did not use the most used MMSE in the models as it is part of the diagnostic criteria of the preclinical stages of AD.

The procedures of these assessments are explained in detail in the ADNI General Procedures Manual (http://adni.loni.usc.edu/wp-content/uploads/2010/09/ADNIGeneralProceduresManual.pdf).

Neuroimaging biomarkers

Amyloid PET imaging data used in this study were provided to the shared ADNI datasets by William Jagust and group at Helen Wills Neuroscience Institute, UC Berkeley & Lawrence Berkeley National Laboratory. The imaging was done using florbetapir data, acquired 50–70min post injection and is expressed as a standardized uptake value ratio (SUVR). As recommended by the group [29], we used a summary SUVR based on the whole cerebellum reference region as an indicator for Aβ aggregation. Since the threshold for establishing Aβ positivity (Aβ+) depends on the tracer and the image processing methods used [30, 31], we used the threshold as PET SUVR>1.11 [32] for amyloid positivity, as suggested by the group that provided the data.

MRIs collected from different locations of the ADNI study were processed by Schuff & Weiner group at the University of California-San Francisco using the FreeSurfer software package (available at http://surfer.nmr.mgh.harvard.edu/); the technical details of this processing are described elsewhere [33–35]. MRI data was collected as a part of screening, some time before the rest of the “baseline” characteristics were measured. For this study, the volumetric measure of hippocampal region (HV) was used. It was reported in previous studies of prediction models in preclinical stages that HV alone is the most useful structural measure [36, 37]. HV was adjusted and normalized for total intracranial volume (TICV). The adjusted volume (HVa) of the region was calculated as HVa = (HV/TICV) * HVpopulation-mean].

Biofluid biomarkers

CSF biomarkers:

The CSF samples collected from ADNI cohorts were analyzed by the ADNI Biomarker Core lab at University of Pennsylvania (UPenn). The analyses of Aβ₄₂, tau, p-tau measures were done using the Roche Elecsys amyloid-β(1–42) CSF, Elecsys Total-Tau CSF, and Elecsys Phospho-Tau(181P) CSF immunoassays respectively, following a Roche Study Protocol, the details of which were described in previous studies [38–40]. For the current study, we used the continuous pg/mL values of CSF-Aβ₄₂, tau, and p-tau. From the most recent data source for CSF biomarkers on LONI website, very few had CSF Aβ_42/40 ratio (N = 216 total, much fewer when considering those with available PET data).

Plasma biomarkers:

The analysis of the tau protein phosphorylated at threonine 181 (P-tau181) and axonal protein neurofilament light (NfL) from ADNI cohorts was done by the Clinical Neurochemistry Laboratory at the University of Gothenburg, Sweden. The analysis was done using the Single Molecule array (Simoa) technique with an in-house assay, the details of which are described previously [41]. For this current study, we used the continuous pg/mL values of p-tau and NfL. From the most recent data source for Plasma Aβ on LONI website, across all diagnostic groups with PET imaging, approximately 200 people had Aβ₄₂ and Aβ_42/40 from their baseline visit.

Statistical analysis

We applied a data-driven approach using machine learning models and baseline data to classify persons with MCI into two classes: one that is closer to cognitively normal individuals (CN-like) and another that is closer to individuals with AD dementia (AD-like, Aβ+ individuals with dementia diagnosis). Subsequently we assessed performance of these models by analyzing their ability to predict incident AD-dementia based on follow-up data. We ran the classification algorithm with different feature-sets to find the relative ability of biomarkers from each of the modalities (see Model Development) to predict MCI conversion. For classification, we used Random Forests (RF), an ensemble machine learning algorithm which also provides the relative importance of the features used for classification (see Model Refinement). We refined each of these models to identify only those features that contribute to the predictive performance of the model (see Model Refinement).

Model development

Predictive features:

All the features considered in this study broadly fall into the following four categories that include different biomarkers. The Demographics and APOE4 category included age, sex, education, race/ethnicity, marital status and number of APOE ε4 alleles (0, 1, or 2). The Plasma biomarkers included Plasma p-tau (P-tau181) and Plasma NfL. The CSF biomarkers included Aβ₄₂, tau, and p-tau181. Finally, MRI measures included the HVa.

Using a combination of these categories, we built six feature-sets: First feature-set is Demographics and APOE4 (DA), which was also included in all the other feature-sets. Three feature-sets were obtained by combining Plasma, CSF, and HVa with DA individually (DA + P, DA + C and DA + H). The fifth feature-set was obtained by combining Plasma and CSF with DA (DA + P + C). The final feature-set was obtained by combining all of Plasma, CSF and HVa with DA (DA + P + C + H). See Supplementary Figure 1.

Classification models:

We used Random Forests (RFs) for classification and prediction. RF is an ensemble learning method that can be used for both classification and regression tasks. RF models train on different samples of the data and using a random subset of features, which makes them suitable for datasets with missing features. RF classifier consists of a multitude of decision trees constructed using the training data, where the output can be the class which most trees choose given an input [42]. RFs are known for their ease of use and inbuilt methods to determine the importance of features in the model.

Model training:

The classification models were trained to classify CN-s and AD groups. A 5-fold cross-validation was used for internal validations. Trained models were used to classify the participants in independent sample of MCI participants. Each MCI individual was classified into CN-like and AD-like groups. The accuracy of the predicted come was evaluated using their diagnosis at the 2-year follow-up visit (i.e., progressed to Dementia or not), considering reasonable timelines for clinical trials and for more likely direct comparison between different studies. Of note, since the outcome of interest, progression to Dementia, does not depend on amyloid pathology, both CN-s groups and MCI group contain both Aβ− and Aβ+ individuals.

Model refinement

RF models provide the relative importance of the features used after fitting the data. For each feature-set, the training phase of the model development involved the following steps:

Model fitting and feature importance:

We fit the training data with all the features in each feature-set and obtain the relative importance of the features from the RF classifier. We used Permutation Importance (PI) to rank the variables in the order of their importance in the model which has been shown to be particularly useful with nonlinear estimators like RF [43]. RF classifier provides a default feature importance which uses Gini importance mechanism (mean decrease in impurity). However, Gini index is known to have limitations like inflating the importance of numerical features, even for the features that are not predictive of target variables. The use of PI can mitigate these limitations as well as being model-agnostic [42]. For this particular study, to train the models, we used data with two groups: CN-s and AD. The trained models are then used with the data from MCI group to assess the prediction to AD-dementia. In this prediction phase, no other intermediate classes are generated. The benefit from the training phase is a model that has learned to classify CN-s group from AD and this model is then applied on the MCI data to obtain the likelihood of an individual belonging to CN-like or AD-like group.

Predictive features (PFs):

We used a forward-selection process. We start with a reference model consisting only of only the variable with highest feature importance. We then build the subsequent models by adding one variable at a time in decreasing order of their importance obtained in Step 1 and repeat the training (Feature Weight in Supplementary Table 3). We retain a variable as a PF if it improves the performance of the model compared to that of the reference model that includes the predictive features selected so far. The criteria for improvement over the reference model is that the area under the ROC curve (AUC) increases by at least 1%.

Feature engineered (FE) model:

Finally, we use the feature engineered model that consists of PFs from the previous step to predict the future outcome in the MCI subgroup.

Reference models

Performance of models and the accuracy of the predicted outcomes for the MCI population—namely, likely to remain stable versus likely to progress to dementia—was evaluated using available longitudinal data. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each model were calculated. Considering change in available longitudinal data (due to dropouts, death, etc.), the performance of models is reported separately for each follow-up at 2 and 4 years.

RESULTS

Study characteristics

Out of the 734 MCI participants who had a 2-year follow-up, 492 had amyloid PET scans. These MCI participants had an average age 71.9 years 44.1% were female, 91.5% were non-Hispanic white, 48.4% had at least one APOE ε4 allele, and 56.2% were Aβ+ (see Supplementary Table 1). Among these MCI participants, 193 (26.3%) during 2 years and 140 (34.4% of those with 4-yr follow up visit) during 4 years of follow-up progressed to dementia.

Model training

We first trained and validated each of the 7 models derived from different feature-sets and obtained cross-validation metrics. Training metrics of the models that included all the corresponding features are shown in Table 2. Model D which included just the demographics had an AUC of 0.58 ± 0.02. Model DA which included Demographics and APOE4 had an AUC of 0.69 ± 0.02. Models DA + P, DA + C, and DA + P + C had higher AUCs than model DA (0.75 ± 0.03, 0.81 ± 0.02 and 0.83 ± 0.03, respectively). Models DA + H and DA + C + P + H, which included HVa had the highest AUCs of 0.85 ± 0.02 and 0.89 ± 0.02, respectively. To compare performance of models using biomarkers as predictors with models using neuropsychological test scores as predictors, we also trained a DA + NP model, which included DA and neuropsychological assessments. This model had an AUC of 0.95 ± 0.01.

Table 2.

CN-s and AD groups: Training metrics of models with all features

N (Base Prevalence)	Model	SEN (%)	SPE (%)	PPV (%)	NPV (%)	AUC (err)
254 (38.19%)	D	46.5	65.1	46.3	65.9	0.56 (0.03)
	DA	58.6	80.2	64.4	76.3	0.69 (0.02)
	DA + P	64.8	86	73.7	80.5	0.75 (0.03)
	DA + C	77.3	84.7	76.1	85.9	0.81 (0.02)
	DA + H	80.6	89.2	82	88.3	0.85 (0.02)
	DA + P + C	78.3	87.3	79.1	86.9	0.83 (0.03)
	DA + C + H	86.6	93.0	88.7	92.0	0.90 (0.01)
	DA + P + C + H	86.5	92.4	87.9	92.1	0.89 (0.02)
237 (34.18%)	DA + NP*	92.6	97.4	95.5	96.3	0.95 (0.01)

BP, base prevalence; D, only Demographics (age, sex, education, race, marital status); DA, Demographics and APOE4; P, Plasma pTau, NFL; C, CSF Aβ₄₂; tau, p-tau; H, adjusted hippocampal volume; NP, NeuroPsych; SEN, Sensitivity; SPE, Specificity; PPV, positive predictive value; NPV, Negative Predictive Value; AUC, Area under ROC Curve; Base Prevalence (%), Base prevalence of Dementia diagnosis in the sample that included CN-s and AD groups.

Feature Selection: Training Phase

During the training phase, to obtain the FE models for each of the feature-sets, feature selection was done after the models were fit with the training data. The first feature in each final model was the most important feature obtained by the PI and subsequently, those features were added to the FE model if the resultant model increases the AUC by at least 1%. This feature selection process led to the following FE models (Supplementary Table 3):

• Model D started with AGE as the most important feature with an AUC of 0.57 ± 0.03. Adding sex improved the performance of the model across most of the metrics, with the final model having an AUC of 0.58 ± 0.02.

• Model DA started with APOE4 as the most important feature with an AUC of 0.68 ± 0.04. Adding age, marital status and then ethnicity/race improved the performance of the model across all the metrics, with the final model having an AUC of 0.76 ± 0.02.

• Model DA + P started with plasma p-tau as the most important feature with an AUC of 0.66 ± 0.03. Adding APOE4 and age improved the performance of the model across all metrics, with the final model having an AUC of 0.77 ± 0.05.

• Model DA + C started with CSF Aβ₄₂ as the most important feature with an AUC of 0.71 ± 0.01. Adding CSF p-tau improved the performance of the model to AUC 0.8 ± 0.01, with the final model with few of the demographic variables reaching only a slightly higher AUC of 0.84 ± 0.01.

• Model DA + H started with HVa as the most important feature with an AUC of 0.77 ± 0.03. Adding age, education, and marital status incrementally improved the performance of the final model (AUC = 0.87 ± 0.01).

• Model DA + P + C, in which all the available features from both plasma and CSF were considered, started with CSF Aβ₄₂ as the most important feature with an AUC of 0.71 ± 0.01. Adding CSF p-tau improved the performance of the model considerably to an AUC of 0.79 ± 0.01. Further adding education and marital status demographic variables improved the model performance, with a final AUC of 0.85 ± 0.01.

• Model DA + P + C + H, in which all the feature-sets from the first 4 models were considered, started with HVa as the most important feature with an AUC = 0.77 ± 0.03. Adding CSF Aβ₄₂ and p-tau improved the performance of the model considerably to an AUC of 0.90 ± 0.01. Adding age and APOE4 improved the final model performance up to an AUC of 0.93 ± 0.01.

Figure 2 highlights the features that are included in each model and those that remained in the FE models. Details about contribution of individual features to the models are provided in Supplementary Table 3.

Fig. 2. Feature selection by model with different combination of biomarker measures.

FE Model, Feature Engineered Model; DA, Demographics and APOE4; P, Plasma; C, CSF; H, Adjusted Hippocampal Volume.

Performance of prediction models after 2 years of follow-up

FE models from each of the feature-sets except that of the model D were used to predict the progression to AD from MCI. Performance of models was assessed using two years of follow-up data. Model DA had a low AUC (0.54 ± 0.02) with sensitivity of 46.9% and specificity of 61.0%. Model DA + P, which included plasma biomarkers in addition to features in base model, had an AUC of 0.64 ± 0.03, with sensitivity of 53.2%, specificity of 74.8%. Model DA + C, which included CSF biomarkers in addition to features in base model, and had the AUC of 0.64 ± 0.03, same as that of Model DA + P, and sensitivity of 57.5%, specificity of 70.9%. Model DA + H, which combined HVa with DA, had an AUC of 0.70 ± 0.03 and sensitivity of 72.6%, specificity of 67.2%.

Model DA + P + C, which combined plasma and CSF biomarkers with DA, had an AUC of 0.66 ± 0.02 and sensitivity of 62.2%, specificity of 70.0%. Finally, Model DA + P + C + H, which combined all the three biomarkers with base model, had the AUC of 0.76 ± 0.03 and sensitivity of 76.1%, specificity of 75.0%. This model with all biomarkers had a higher predictive performance across all metrics than the model DA + NP, which had an AUC of 0.72 ± 0.02. A detailed overview of the performance metrics of the models can be found in Table 3 (also see Supplementary Figure 2).

Table 3.

MCI population:Prediction metrics (AD-like versus CN-like after 24-month followup) of Feature Engineered models

Model	N	SEN (%)	SPE (%)	PPV (%)	NPV (%)	AUC (err)	Base Prevalence (%)
DA	728	46.9	61	30.1	76.2	0.54 (0.02)	26.4
DA + P	363	53.2	74.8	30.3	88.6	0.64 (0.03)	17.1
DA + C	460	57.5	70.9	42.9	81.4	0.64 (0.03)	27.6
DA + H	414	72.6	67.2	32.1	92	0.70 (0.03)	17.6
DA+P+C	460	62.2	70	44.1	82.9	0.66 (0.02)	27.6
DA+C+H	313	76.1	76.8	47.2	92.2	0.76 (0.03)	21.4
DA+P+C+H	311	76.1	75	45.5	92	0.76 (0.03)	21.5
DA + NP	732	75.1	68.8	46.3	88.5	0.72 (0.02)	26.4

DA, Demographics and APOE4; P, Plasma; C, CSF; H, adjusted hippocampal volume; NP, NeuroPsych; SEN, Sensitivity; SPE, Specificity; PPV, positive predictive value; NPV, Negative Predictive Value; AUC, Area under ROC Curve; Base Prevalence (%), Base prevalence of Dementia diagnosis in the sample using longitudinal data at 24 months.

Performance of models with all the features

Models were also tested with all the available features in their feature-sets without any feature engineering. Overall performance of models with all measures were comparable to the corresponding model after feature selection (Supplementary Figure 3, Supplementary Tables 2 and 3). For each of the models, to validate that there is no statistically significant difference between the performance of each model and its FE counterpart with fewer, selected features, we performed McNemar Test [44], which uses a continuity corrected chi-squared statistic [45]. Only model DA showed significant differences between the model with all the features and the FE model (AUCs of 0.59 ± 0.02 and 0.54 ± 0.02 respectively, χ² = 73, p = 0.009). For the other models, predictive performances of their FE versions were not statistically different from their counterparts which included all the features.

Performance of prediction models in MCI-Aβ + population

We tested performance of the prediction models specifically in the Aβ + subset of the MCI population (Table 4). Models DA & DA + P showed lower AUCs of 0.56 ± 0.04 and 0.62 ± 0.04, respectively. Models DA + C and DA + P + C had AUCs of 0.65 ± 0.04 and 0.64 ± 0.04, respectively. Models involving HVa (DA + H and DA + P + C + H) had AUCs of 0.65 ± 0.04 and 0.66 ± 0.04, respectively, with the former showing a higher sensitivity of 63.9% and a higher NPV of 81.5% whereas the latter showed a higher specificity of 80.2% and higher PPV of 54.7% (BP = 31.2%).

Table 4.

MCI-Aβ + population: Prediction metrics (AD-like vs CN-like) of Feature Engineered models

Model	N	SEN (%)	SPE (%)	PPV (%)	NPV (%)	AUC (err)	Base Prevalence (%)
DA	206	57.4	55.2	35	75.5	0.56 (0.04)	29.6
DA + P	193	52.7	71.7	42.6	79.2	0.62 (0.04)	28.5
DA + C	185	51.7	77.6	52.5	77	0.65 (0.04)	32.4
DA + H	206	63.9	66.9	44.8	81.5	0.65 (0.04)	29.6
DA + P + C	176	49.1	79.3	51.9	77.4	0.64 (0.04)	31.2
DA + C + H	186	55.0	76.2	52.4	78.0	0.66 (0.04)	32.3
DA + P + C + H	176	52.7	80.2	54.7	78.9	0.66 (0.04)	31.2
DA + NP	205	61.7	80	56.1	83.5	0.71 (0.04)	29.3

DA, Demographics and APOE4; P, Plasma; C, CSF; H, adjusted hippocampal volume; NP, NeuroPsych; SEN, Sensitivity; SPE, Specificity; PPV, positive predictive value; BP, Base Prevalence; NPV, Negative Predictive Value; AUC, Area under ROC Curve; Base Prevalence (%), Base prevalence of Dementia diagnosis in the sample using longitudinal data at 24 months.

Performance of prediction models with higher cut-off for Aβ positivity

The primary analyses have been performed considering the Aβ positivity as PET SUVR > 1.11 [32]. However, several studies have explored the performance of different cut-offs of PET SUVR for Aβ positivity. We analyzed the models with a different threshold of 1.19 that was shown to be suitable for longitudinal analyses [46, 47]. The performance of the models in predicting MCI-to-AD conversion was similar to the primary analyses (Supplementary Table 5)

Performance of prediction models after 4 years of follow-up

We repeated all the analysis using the criteria for CN-s group as CN individuals who remained CN during 4 years of follow up and the new outcome of progression from MCI to Dementia during 4 years of follow up. Performance of these models are summarized in Supplementary Table 4 and Supplementary Figure 4.

DISCUSSION

This study provides a comprehensive overview of predictive ability of different combinations of biomarkers in predicting the disease progression from MCI to dementia and Alzheimer’s dementia. We showed that the combination of MRI, CSF and plasma biomarkers performs better than any of the biomarkers individually—both in classifying individuals between AD and CN at baseline as well as in predicting progression from MCI to dementia. Furthermore, our results also show that adding at least one biomarker measure to demographics and APOE ε4 information significantly improves the ability to classify individuals between AD and CN, and to predict the progression of MCI to dementia. In the context of using blood-based biomarkers for prediction disease progression in prodromal stages of AD, our results highlight that plasma biomarkers performed similarly to CSF biomarkers, emphasizing on the utility of their low-cost and minimally invasive nature. Overall, the modelling approach we employed provides the ability to understand the diagnostic and predictive abilities of specific biomarkers with in each modality.

Novel BBMs have huge potential to revolutionize the field of ADRD. This is due to their minimally invasive nature, higher accessibility, and lower costs in comparison with currently established imaging (PET, MRI) and CSF biomarkers. But to be able to use them in real world research and clinical practice, we need to establish their diagnostic and prognostic value in ADRD. Many recent studies showed the diagnostic abilities of several plasma and serum biomarkers in AD [12, 48–50] and other dementias [51, 52]. Studies have shown the ability of plasma p-tau to differentiate individuals with AD from cognitively normal individuals [19]. In addition, it was also shown that adding plasma p-tau and NfL to reference models that included demographics and APOE ε4 status improved the ability to predict progression to AD in MCI individuals [53]. Therefore, one of the aims of our study was to assess the relative ability of plasma biomarkers to predict disease progression in comparison with the gold-standard MRI and CSF measures. Although we trained a model with only demographics, model D, it was evident that the model DA with APOE ε4 status included performed a lot better than just demographics. APOE ε4 allele carriership is associated with a shorter duration of the disease stages [54]. Given that our study focuses on disease progression in prodromal stages, for further analyses, we decided to consider DA as our reference model to have subsequent comparisons while adding individual biomarkers. The reference model DA included demographics and APOE ε4 status, as several studies have consistently shown their high performance in predicting amyloid positivity [37, 55–57] (in combined population including all three groups—cognitively normal, MCI, and AD-Dementia). In line with such previous studies, we showed that models that included plasma biomarkers, DA + P, DA + P + C, and DA + P + C + H, improved the overall performance of the models compared to those without the plasma measures, DA, DA + C, and DA + H, respectively. Our results from the Feature Engineered model of DA + P particularly highlighted the relative importance of plasma p-tau over NfL in classifying individuals into AD and CN-s groups and predicting MCI progression to dementia, which is in line with results from previous studies where adding NfL did not improve the performance of the prediction models [19, 58]. Although BBMs show slightly lower predictive ability than CSF or MRI measures alone, given low-cost, low-burden nature of measuring BBMs, they could be highly valuable for diagnosis, effective recruitment of trials or as trial outcome measures.

Our results were also largely in agreement with previous machine learning based models that used only one of CSF, MRI measures or neuropsychological tests [21, 22, 59]. These studies have reported AUCs of 0.68–0.76 [21, 22, 60]. Studies using CSF biomarkers as predictors reported AUCs of range 0.57–0.67 [21, 22, 59, 61]. However, we highlight the ability of baseline plasma biomarkers in particular in predicting the disease progression. Recent studies have shown the power of plasma p-tau181 in the prediction of progression to dementia in individuals with MCI in different datasets [18, 19]. We not only confirm these results in a multi-center study such as ADNI, by showing that models with plasma p-tau181 perform comparable to those with CSF measures, but also present the comparative performances of models using CSF and MRI in addition to plasma measures. In addition, we demonstrate that further understanding can be gained by understanding the relevance of specific biomarkers within each of the modalities using the feature engineering approach.

There is a common misperception that having more data will always impact diagnostic or prognostic ability in a positive way. However, as it has been shown in many other predictive analytic studies [7], simply having more data is not always useful. Primary aim of data collection should be to obtain more “informative” data (higher signal-to-noise ratio). In fact, the added burden imposed by collection of additional data might offset their benefits. Our findings are largely in agreement with this theory. We showed that Feature Engineered models have similar performance with the models that use all features. Overall, these findings indicate that when our aim is to predict disease progression (i.e., prognostications), access to a select number of predictors would be sufficient. Machine learning models have been proven to be effective for diagnostic disease prediction [62, 63]. In the case of AD-dementia, several models have been developed to classify multiple stages: CN from MCI, CN from AD-dementia, and MCI from AD-dementia [64, 65]. But it has been shown that differentiation of MCI and the other participant groups appeared be the most challenging among the diagnosis groups [66]. Multiple models identified the markers of cognitive function in different stages of dementia which might be the signs of disease advancement [67–69]. Therefore, we trained our models with the CN and AD-dementia participants in conjunction with their longitudinal follow up diagnosis information thereby accounting for the temporal patterns and changes associated with disease progression, as shown earlier in multi-stage classification models in AD [70, 71] and minimizing the chance of any overfitted predictive models. Furthermore, the validation of the use of these models in the MCI participants using the known longitudinal outcomes of individuals who progressed from MCI to AD-dementia provides a robust evaluation of the models’ performance [63].

Similar to our findings, many other studies indicate that inclusion of two or more classes of biomarkers provide the highest predictive performance [21, 22, 72]. Many studies that achieved high performance in predicting MCI progression to dementia concurrently included neuropsychological scores and biomarkers as predictors [7]. However, our group has previously showed that neuropsychological scores alone might have similar predictive value to biomarkers [57], which is likely in-part because clinical outcomes are also defined by neuropsychological scores (i.e., predictor and outcome being from the same class of information). Therefore, to better understand predictive value of each class of biomarkers, we separated neuropsychological scores from biomarkers in feature engineering stage. In line with the other findings, our results confirm that models with multiple biomarkers performed better than models with any individual biomarkers in predicting MCI progression to dementia with a PPV of 23.3–26.2% higher than the base prevalence of clinical progression and an NPV of 12.2–13.5% higher than percentage of participants who remained stable during follow up. To our knowledge, our study is one of the first to assess volumetric MRI alongside the latest hallmark AD biomarkers in CSF and plasma without neuropsychological scores, while recent comprehensive reviews of prediction models for conversion from MCI to AD-Dementia highlights most studies that include neuropsychological scores and do not include plasma biomarkers [73]. In addition, the model DA + P + C + H, which combined all the biomarkers (plasma, CSF, and MRI), performed on par with the model that only included neuropsychological tests in addition to demographics (DA + NP). Neuropsychological tests are very useful in predicting cognitive decline and disease progression in AD. However, despite being low-cost and non-invasive, they are time taking for both subjects and the experts alike. Therefore, the performance of plasma biomarkers which are less time consuming must be assessed in comparison to neuropsychological tests.

This study has limitations. Out of the 700 participants who were cognitively normal at baseline and of the 963 participants who had a diagnosis of MCI at baseline, there was no 2-year follow up data for 131 participants and 229 participants respectively. Loss to follow up in cohort studies is unavoidable and sometimes considerable [74], possibly undermining the value of the study [75]. We included only a few biomarkers within each of MRI, CSF, focusing on the most relevant biological biomarkers based on literature to keep the models simple and practical for real-world use. However, using additional features (e.g., regional brain volumes) might improve predictive performance of models. There are other emerging plasma biomarkers, (e.g., Aβ₄₂/Aβ₄₀ ratio, p-tau231) which have been shown to have good predictive value However, data on such biomarkers are limited in current ADNI data base. Future studies are required to evaluate additive value of such biomarkers in prediction of disease progression. Since the primary goal of the study was to compare the performance of models including different biomarker measures, we excluded the participants with missing data in any of the features of the model DA + P + C + H from all the models. The loss of participants to follow up as well as excluding the participants with some missing data certainly impacts the sensitivity of our findings. Another limitation of this study could be regarding the modeling method that is employed. We used Random Forests (RFs), which, as in their family of ensemble machine learning methods, are known for better performance and robustness compared to other individual predictive methods. To establish if the relative performance difference in the findings between the feature-sets is robust, this work can be extended by applying more diverse modeling methods on the same feature-sets. Finally, it should be mentioned that ADNI study is not a population-based study, and therefore generalizability of our findings should be evaluated in other populations. In particular, it has to be noted that ADNI does not specify the subtypes of the Dementia diagnosis. Therefore, we considered the Dementia diagnosis together with amyloid burden to evaluate AD-Dementia.

In conclusion, in this study we showed that machine learning models with diverse feature-sets can be used to classify patients with AD from cognitively normal and to predict disease progression in MCI patients. If our goal is to develop practical predictive models and decision support tools for clinicians or researchers, we should consider limitations of the healthcare system (cost, professional time/effort) as well as burden on patients and caregivers in the data collection phase. Overall, with further development and validation, predictive tools can be used as powerful tools, affecting clinical research and ‘real-world’ clinical decision-making.

DATAAVAILABILITY

The data used in the preparation of this study were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database including its phases 1,2,GO and 3 (https://adni.loni.usc.edu/). The code used for the analyses is available on github (https://github.com/cervere/adni-mci-dem).