Integrating Machine Learning and Environmental and Genetic Risk Factors for the Early Detection of Preclinical Alzheimer’s Disease

Abstract

Objective

This study classified preclinical Alzheimer’s disease (AD) using cognitive screening, neighborhood deprivation via the area deprivation index (ADI), and sociodemographic and genetic risk factors. Additionally, it compared the predictive accuracy of multiple machine learning algorithms and examined model performance with two bootstrapping procedures.

Methods

Data were drawn from a longitudinal cohort that required participants to be age 65 or older, cognitively normal at baseline, and active drivers, defined as taking at least one trip a week. Naturalistic driving data were collected using a commercial datalogger. Biomarker positivity was determined via amyloid pathology using cerebrospinal fluid and positron emission tomography imaging. ADI was captured based on geocoding latitude and longitude to derive a national ranking for the specific location (home or unique destination). Machine learning algorithms classified preclinical AD. Each individual model’s predictive ability was confirmed in a 20% testing dataset with 100 rounds of resampling with and without replacement.

Results

Among 292 participants (n = 2,792 observations), including ADI of trip destinations, participants’ home ADI, and frequency of trips to the same ADI led to a slight but notable improvement in predicting preclinical AD. The ensemble model demonstrated superior predictive performance, highlighting the potential of integrating multiple models for early AD detection.

Discussion

Our findings underscore the importance of incorporating socioeconomic and environmental variables, such as neighborhood deprivation, in predicting preclinical AD. Addressing socioeconomic disparities through public health strategies is crucial for mitigating AD risk and enhancing the quality of life for older adults.

By 2050, the population of older adults (≥age 65) in the United States (U.S.) will double to 88 million, along with an estimated increase of nearly 13 million cases of Alzheimer’s disease (Alzheimer’s Association, 2024). Older age and apolipoprotein (APOE) ε4 are the putative individual-level risk factors for AD; however, race (a social construct) suggests that Black/African American and Hispanic/Latinx are also at a higher risk compared with non-Hispanic Whites (Babulal et al., 2019). In addition to race serving as a social construct, it is a crude proxy marker for racism and health inequities resulting from structural and social determinants of health (S/SDOH). Recent efforts in epidemiological and health sciences recommend replacing race with S/SDOH variables that capture multiple dimensions of the environment using multilevel models (Adkins-Jackson et al., 2022). S/SDOH are upstream factors encompassing social, economic, and political systems that affect conditions, including housing, employment, and education, culminating in a single point (e.g., location) that moderate the risk of developing cognitive decline and AD (Adkins‐Jackson et al., 2023).

Geographical location and neighborhood deprivation are S/SDOH markers associated with home residence but are also linked with poorer cognitive functioning and AD neuropathology (Powell et al., 2020; Zuelsdorff et al., 2020). A recent prevalence study of 3142 counties across the 50 U.S. states found a greater concentration of AD cases in the east and southeastern regions of the United States (Dhana et al., 2023), suggesting there is a relationship between location and prevalence. The area deprivation index (ADI) measures socioeconomic deprivation and serves as a national metric to quantify social and economic disparities at the neighborhood level to understand disease prevalence, mortality rates, and access to health care (Kind et al., 2014; Singh, 2003). The ADI uses 17 weighted indicators (e.g., income, employment, education, housing) based on the U.S. Census and American Community Survey to quantify deprivation at the state and national level at the census block level (Kind & Buckingham, 2018). The ADI has been used in studies to examine relationships between a participant’s residence and cognitive functioning (Vassilaki et al., 2023). A recent study found an association between home ADI and cognitive functioning, where Mexican Americans residing in neighborhoods with more deprivation had poor cognitive functioning compared to non-Hispanic Whites (Wong et al., 2023). Another study of cognitively normal older adults found that older adults racialized as Black had greater ADI, higher blood pressure, body mass index, and white matter hyperintensities compared with their white counterparts. However, the ADI explained 97% of the indirect effects of racial differences in cortical volume (adjusting for demographics and comorbidities; Meeker et al., 2020). Although ADI has been used to study the relationship between neighborhood deprivation with more significant deprivation and a delayed prodromal diagnosis of AD, there is limited research examining the relationship between neighborhood deprivation and preclinical AD.

Preclinical AD is the asymptomatic period (15–20 years) where an adult is cognitively normal, but subtle pathophysiological changes in the brain begin long before clinical symptoms appear. Biomarkers of beta-amyloid (plaques) and tau (tangles) are detected through radiotracers via positron emission tomography (PET) imaging or levels in cerebrospinal fluid (CSF; Vos et al., 2013). The preclinical stage of AD is essential for early detection, understanding proteomic mechanisms, monitoring disease progression, and understanding novel environmental factors contributing to additional AD risk. Recent studies of cognitively unimpaired older adults have found that functional changes in gait performance leading to falls, financial exploitation vulnerability leading to poor financial decisions, and a decline in driving behavior leading to crashes have all been associated with preclinical AD (Bollinger et al., 2024; Fenton et al., 2024; Roe et al., 2017). The changes in these instrumental activities of daily living are early functional markers that portend a subsequent decline that will likely catalyze observable deficits in cognitive functioning measured via neuropsychological tests. To date, limited work has examined the association between preclinical AD and neighborhood deprivation via the ADI as a proxy for S/SDOH.

Measurement via ADI to study neighborhood deprivation is often extracted from self-reports of home addresses, and a handful of frequent locations visited. However, this subjective method is limited and prone to validity errors, including recall, social desirability, and sampling biases. Smartwatches and global positioning devices (GPS)-based sensors can objectively quantify and monitor daily physical activities, cognitive status, physiological function, and overall movement across time and space. Changes in these activities can be early indicators of cognitive decline associated with AD. The continuous data collected from wearables and sensors can be integrated and analyzed using machine learning algorithms to identify patterns, predict the risk of developing AD, and monitor disease progression (Kourtis et al., 2019). Naturalistic driving behavior using dataloggers has been used to assess the functional impact of preclinical AD. Still, it can also be a complementary way to predict AD before a functional decline occurs (Babulal et al., 2021). Spatial meta-data like location (latitude and longitude) collected from daily driving can provide valuable context about interaction with the environment to inform changes in the preclinical stage of AD (Babulal et al., 2016; Roe et al., 2019). Our longitudinal studies in driving, AD biomarkers, and using GPS-based sensors via a commercial datalogger plugged directly into participants’ vehicles collect large volumes of naturalistic data. The locations derived from the GPS allow quantification of each destination’s deprivation based on the ADI.

Given the rapid integration and exponential growth of artificial intelligence (AI) across sectors of health care and medicine, AI applications in gerontological research can support risk prediction and classification of diseases like AD. However, key challenges persist with AI’s lack of transparency in model selection within machine learning, complicating the reproducibility of research findings, as undocumented choices and biases in the selection process of parameters can lead to inconsistent results across datasets and studies. Without clear documentation and justification of model selection criteria, other researchers struggle to replicate and validate findings, undermining the credibility of the research. This opacity also impacts the ability to identify and mitigate potential biases and errors, which can perpetuate the promotion of poor models and unreliable conclusions. The objectives of this study are to (a) classify preclinical AD using common sociodemographic and genetic risk factors, a cognitive screen, neighborhood deprivation of home and frequently visited locations, (b) compare the predictive accuracy of multiple machine learning algorithms for discriminating preclinical AD, and (c) examine model performance with two bootstrapping procedures.

Method

Data Collection and Study Population

Data for this study were drawn from a longitudinal, prospective cohort of participants enrolled in The DRIVES Project at Washington University School of Medicine. The study examined how preclinical AD affects complex neurobehavioral activities like driving. All participants provided written informed consent. The study protocol was approved by the Washington University Human Protection Office Institutional Review Board (201706043, 202010214, 202003209). Eligible participants were required to be (a) at least 65 years of age or older, (b) cognitively normal at baseline as determined by a rating of 0 on the Clinical Dementia Rating^® (CDR^®; Morris, 1993), (c) drive at least once a week, (d) possess a valid driver’s license, (e) have a vehicle with a working onboard diagnostic (OBD-II) port, and (f) willing to complete biomarker testing every two to three years.

Clinical Assessment

The annual visit included surveys and neurological and neuropsychological assessments. Trained clinicians completed the CDR to detect intraindividual change in both cognitive and functional abilities across six different domains (Memory, Orientation, Judgement and Problem-Solving, Community Affairs, Home and Hobbies, and Personal Care). Scores for each domain were used to compute a global rating (0, 0.5, 1, 2, 3), indicating the severity of the level of cognitive impairment with higher ratings. The Mini‐Mental State Examination (MMSE) and the Montreal Cognitive Assessment Test for Dementia (MoCA) were also captured during their visit. Sociodemographic factors were collected at the baseline visit. Cutoffs were used for the CDR (≥ 0.5), MMSE (<24), and MoCA (<26) were recoded to binary (0 vs 1) to denote normal and cognitive impairment.

Naturalistic Driving

Daily driving data were continuously collected using a commercial datalogger (Azuga Inc., San Jose, CA) installed in the OBDII port of a participant’s vehicle. The Driving Real-world In-Vehicle Evaluation System (DRIVES) methodology has been established and validated in a number of previous studies (Babulal et al., 2016). Investigation of driving behaviors as predictors of preclinical AD utilized available driving trip-level data that contained locations of places visited. Only the latitude and longitude at the beginning and ending destinations of each trip were examined for this study, in addition, at least 1 year of data was required for inclusion.

Neighborhood Deprivation

The ADI used data from the U.S. Census Bureau and the American Community Survey to assess common standardized, weighted, and combined indicators like income, education, employment, and housing quality to produce a composite score for each census block (Kind & Buckingham, 2018). Scores are available for state (1–10) and national (1–100) rankings, with higher scores indicating greater deprivation. The 2021 ADI was downloaded from the Neighborhood Atlas (https://www.neighborhoodatlas.medicine.wisc.edu/), a website maintained and updated by the University of Wisconsin School of Medicine and Public Health. The participant's home addresses and trip destinations were identified and linked to the ADI for this study and included in the analysis as the primary predictors.

Biomarker Positivity

The primary outcome of this study was the participant’s preclinical AD status (positive vs negative) based on either CSF or PET imaging amyloid radiotracers. CSF was collected as described in previous studies (Fagan et al., 2006). Amyloid-beta₄₂ (Aβ₄₂)/amyloid-beta₄₀ (Aβ₄₀), (Lumipulse G1200, Fujirebio, Tokyo, Japan) was used for determining the presence of preclinical AD as the outcome (cutoff: Aβ₄₂/Aβ₄₀ <0.0673) (Volluz et al., 2021). In addition to the CSF values, apolipoprotein genotyping was conducted to determine whether a participant carried at least one copy of the ε4 allele, which was then classified as APOE ε4 carrier versus noncarrier. PET amyloid imaging was completed with either Pittsburgh Compound-B ([¹¹C] PiB) or florbetapir (F-AV-45) tracer. Binding potentials and standardized uptake value ratios were calculated utilizing the (30–60 min for PiB; 40–70 min for F-AV-45) post-injection window using the cerebellar gray matter as the reference region. Global amyloid burden was represented by averaging regions that are known to be sensitive to AD pathology together. Amyloid was classified as positive if the mean cortical SUVR regional spread function was greater than 1.42 (Su et al., 2015) for PiB and greater than 1.19 for F-AV-4 5(Su et al., 2019) tracers.

Data Management

Our key predictor variables included demographics (age, sex, education), clinical (CDR, MoCA, MMSE), genetic variables (APOE ε4 positivity), and geocoded location (ADI). We limited our sample to participants with a biomarker outcome and nonmissing information in any identified predictors for this study. The date of initial datalogger installation, or plug-in date, represented the index date for each participant. Age was computed as the difference between the plug-in date and birth year. For participants with multiple cognitive tests for each individual test, we included the cognitive test scores closest to the index plug-in date (minimum absolute difference between the index date and date of cognitive testing). Similarly, if a participant was determined to have more than one “status” throughout the study, the “status” at the time point closest to the index plug-in date was selected as the participant study outcome (minimum absolute difference between the index date and the date of status). Additionally, the time difference between this index date and the outcome, the CDR, the MMSE, and the MoCA were also included as variables in the model. Finally, a threshold was used to categorize participants into preclinical AD positive vs negative participants, given their biomarker status.

Federal Information Processing Series (FIPS) codes were derived from the longitude and latitude driving data using the tigris R package in R (Walker, 2016) and used to geocode both the home addresses as well as their trip destinations. We then used the FIPS codes to link addresses and trip destinations to the 2021 ADI data, prioritizing the national ADI rank given the proximity to Illinois, participants residing in both states, and the heterogeneity of routes traveled between both states. For each home and destination variable, we grouped participant's ADI data into deciles based on their percentile, with each rank representing 10% of the data. A total number of 18,262 trips were identified for 292 study participants. This limited the longitudinal dataset for the 292 participants to 2792 unique observations. We also computed the sum of the trips each participant made to a specific destination within each 10th percentile for all participants and included it as another predictor.

Statistical analysis

Unique patient identifiers in the sample of 2,792 observations were randomly split into a model derivation cohort (80%, N = 2,233) and a testing set (20%, n = 559). The derivation set was locked until the final model was derived. To evaluate whether the data splitting resulted in a balanced classification of the outcome and predictors across the training and testing subsets, we conducted statistical testing using t-test and Chi-square for continuous and categorical variables, respectively. Additionally, correlations of each variable with the outcome were evaluated in the training and testing subsets.

Variable Selection

For variable selection, we implemented built-in methods across seven models from Lasso in logistic regression (LR), Support Vector Machines (SVM), Classification and Regression Trees (CART), a Random Forest Classifier (RFC), extreme gradient boosting (XGBoost), ensemble and Deep Learning (DL) models to identify all relevant variables to predict presence or absence of preclinical AD. All methods are designed to iteratively remove variables proven to be less relevant than random probes. Variable selection was made in two stages. First, variables of interest (e.g., ADI for neighborhood deprivation of home addresses and trip destinations) were included in the model regardless of their score on the feature importance analysis. Among collinear variables (i.e., cognitive tests), we chose one with the strongest correlation with the outcome. We noted that the magnitude and direction of the correlation were consistent with prior literature (Roe et al., 2018). For the remaining variables, we relied on a combination of factors, including their scores in the feature importance analysis, their clinical relevance, and the direction of their association with the outcome (i.e., if consistent with the current literature). The selection method identified 10 variables we used in model derivation (see Supplemental Section). Accuracy was the evaluation metric examined as the primary outcome, where higher values indicated a better fit for the overall model.

Machine Learning Algorithms

We implemented LR, SVM, CART, RFC, XGBoost, and Deep Learning (DL) algorithms that examined the overall accuracy and parameters that are germane to ML models (accuracy, precision, sensitivity, F1 score, Kappa, specificity, negative predictive value, balanced accuracy) along with a ranking over the overall strength. To improve the overall accuracy and robustness, we employed an ensemble model to leverage the strength of the tested ML algorithms. Specifically, we combined the outputs of LR, SVM, CART, RFC, XGBoost, capturing the unique perspectives of these base models with the data, along with different patterns and nuances. The stacked ensemble model averages these individual models’ predictions to derive a meta-estimator that combines weighted predictions of different ML algorithms to return an optimized average predicted value, leading to a more balanced and reliable outcome. This averaging process helped mitigate any single model’s biases, improving generalization to new data. Therefore, the ensemble stacked model represented an approach that enhances the predictive performance by integrating the complementary strengths of the machine learning techniques applied in this study (Ting & Witten, 1997). We applied that approach to the ML models developed using three approaches (linear, bootstrap with, and without replacement) in both the training and testing subsamples. In the DL models, we employed the rectified linear unit activation function in the input layer to introduce nonlinearity, facilitating the model’s ability to capture complex patterns within the data. The input layer consisted of 32 units, corresponding to the number of neurons. Two hidden layers were added to the model architecture to enable the model to learn hierarchical representations of the input data. A single-unit output layer with a sigmoid activation function was utilized, which is suitable for binary classification tasks. Stochastic Gradient Descent was chosen as the optimizer. To enhance training dynamics and ensure optimal convergence, a learning rate scheduler was incorporated to promote an adaptive learning rate that prevents overshooting and oscillations.

Model Derivation

In the 80% training sample, we conducted model derivation using the three approaches in consideration of the longitudinal format of the data set with multiple trip-related ADI information. In the first approach, we fed the ML models with data in a linear format without consideration of the participants’ intra-individual correlation. In the second method, we used bootstrapping with replacement to generate 100 data sets, fit a model in each, and summarize the evaluation metrics. In the third approach, we used bootstrapping without replacement, allowing for one ID per participant to be selected for each bootstrapped data set and fit a model in each to derive an evaluation metrics profile. This last approach accounted for the intra-individual correlation among participants.

Model Validation

We used the 20% set aside sample to quantify the accuracy of each of the algorithms using the accuracy score, recall, F1 score, precision, kappa, specificity, negative predictive value, and balanced accuracy. These metrics with associated terms, definitions, and calculations can be found in Table 1. Comparisons of the scores in the testing sample were plotted for all ML models. We used a line graph to generate a plot of the overlayed evaluation metrics from each of the ML models for visual comparisons.

Table 1.

Evaluation Metrics and Terms With Common Synonyms, Definitions, Purpose, and Calculations

Variable	Synonyms	Definition	Purpose	Calculation
Evaluation metrics
Accuracy score	N/A	Total ratio of correct predictions	Demonstrate model’s overall correct class predictions	(TP + TN) / (P + N)
Balanced accuracy	BA, balanced classification rate (BCR), macro F1, macf1	Mean of true positive and true negative rates	Demonstrate model’s class-aware correct predictions	(TPR + TNR) / 2
Recall	Sensitivity, true positive rate (TPR)	Rate of true positives to all positive truth values	Reduce type ii errors in the model	TP / (TP + FN) or TP / P or 1—FNR
Specificity	Selectivity, true negative rate (TNR)	Rate of true negatives to all negative truth values	Reduce type i errors in the model	TN / (TN + FP) or TN / N or 1—FPR
Precision	Positive predictive value (PPV)	Ratio of true positives to all predicted positives	Demonstrate model’s correct positive class prediction	TP / (TP + FP) or TP / PP or 1—FDR
Negative predictive value	NPV	Ratio of true negatives to all predicted negatives	Demonstrate model’s correct negative class prediction	TN / (TN + FN) or TN / PN or 1—FOR
Kappa	Cohen’s kappa coefficient, κ	Statistic measuring the degree of agreement between raters	Demonstrate model’s performance compared to randomized accuracy	2 * (TP * TN—FP * FN) / ((TP + FP) * (TN + FP) + (TP + FN) * (TN + FN))
F1 score	N/A	Harmonic mean of precision and recall	Demonstrate precision and recall as a single value	(2 * TP) / (2 * TP + FP + FN) or (2 * PPV * TPR) / (PPV + TPR)
Terms
Positive	P	Total positive truth values		TP + FN
Negative	N	Total negative truth values		N + FP
Predicted positive	PP	Total predicted positive values		TP + FP
Predicted negative	PN	Total predicted negative values		TN + FN
True positive	TP	Correctly assigned positive prediction		Where predicted P = truth P
False positive	FP	Incorrectly assigned positive prediction		Where predicted P!= truth N
True negative	TN	Correctly assigned negative prediction		Where predicted N = truth N
False negative	FN	Incorrectly assigned negative prediction		Where predicted N!= truth P
True positive rate	TPR	See “recall”		See “recall”
False positive rate	FPR	Rate of false positives		FP / (TN + FP) or FP / N or 1—TNR
True negative rate	TNR	See “specificity”		See “specificity”
False negative rate	FNR	Rate of false negatives		FN / (TP + FN) or FN / P or 1—TPR
False discovery rate	FDR	Rate of false positives to all predicted positive values		FP / (TP + FP) or FP / PP or 1—PPV
False omission rate	FOR	Rate of fale negatives to all predicted negative values		FN / (TN + FN) or FN / PN or 1—NPV
Population	N/A	All predictions		P + N

Results

Description of the Study Sample

Summary statistics for each predictor are illustrated in Table 2. Bivariate analysis between the independent predictors and the outcome variable (preclinical AD) revealed one statistically significant association with the MoCA (p = .045). Correlations from strongest to weakest are shown in Figure 1. None of the numeric variables included any outliers, and as a result, no data were manipulated to eliminate potential outliers. The distributions for these variables were not found to be normal. Notably, the educational distribution, the difference in days between the MoCA test and the plug-in date, and the number of trips made to the same ADI rank were all negatively skewed. Preclinical AD status revealed a class imbalance with a 1:4 ratio, with the number of positive cases (n = 752) representing 26.4% of the sample. Among the individual predictors, age, APOE, home ADI rank, number of trips to the same ADI rank, MoCA, and difference in days between the MoCA test date and the plug-in data showed significant differences across the outcome. Penalized logistic regression using Lasso confirmed 10 attributes as important for further modeling (Table 2).

Table 2.

Exploratory Analysis of the Independent Variables (N = 2,792)

Predictora	Median [IQR]	N (%)	p-Value, statistical test b	Lasso coefficient shrinkage c	Score
Age (years)	72.4 [69.2, 76.2]		.915	5	0.009376
Sex			.597	10	0.000000
Female		1,540 (54.1%)
Male		1,309 (45.9%)
Education (years)	16.0 [15.0, 18.0]		.102	1	−0.097889
APOE ε4+		922 (33.0%)	.388	3	0.071997
ADI national rank	5.0 [2.0, 7.0]		.584	6	−0.007742
Number of trips to same ADI	85.0 [26.0, 217.0]		.256	9	0.000231
Home ADI national rank	4.0 [2.0, 5.0]		.212	2	−0.084014
Difference in time between outcome and plugin date (days)	268.0 [146.0, 339.0]		.203	8	−0.000425
MoCA
Dichotomous		1,618 (56.8%)	.046	4	0.071232
Continuous	26 [24, 28]		.191
Difference between MoCA and outcome date (days)	175.0 [70.0, 366.0]		.255	7	−0.001531

Notes: IQR = Interquartile range; APOE = apolipoprotein; ADI = Area Deprivation Index; MoCA = Montreal Cognitive Assessment. For each independent variable, there were no statistically significant baseline differences between the train and test subsets in the dataset (except for MoCA).

^aTesting for association between predictors and status. Respective binary classifier importance following implementation of the penalized regression coefficient shrinkage for each predictor by the Lasso, variable selection by Random Forest Classified, Support Vector Machine, XGBoost, and CART, built-in methods.

^bBivariate analysis: testing for association between an individual predictor and the response variable (Chi-squared p-values).

^cPooled results after Lasso was utilized for variable selection.

Figure 1.

Correlation heat map of predictors

Logistic regression

When comparing the LR models with penalized regression using Lasso with the whole data set and the bootstrapped data with replacement and without replacement, there was no difference in the predictive performance compared with the rest of the models. Performance improved following regularization using Lasso and the application of scaling. We achieved a pooled accuracy of 74.2% (95% confidence interval [CI]: 74.1%–74.4%) with an aggregated F1 score of 0.225 in the final model. Lasso regression classifiers trained on the bootstrapped with replacement (73.6%) and without replacement (74.3%) were slightly less accurate. All models achieved a higher mean specificity rate but with a further decrease in the recall and F1 score (Table 3). Furthermore, Lasso regression models trained on the entire dataset detected an optimal mean lambda of 0.04, the best value that minimized the cross-validation prediction error. Lasso regression analysis did not demonstrate any superiority in training better predictive models compared to the other models.

Table 3.

Linear Predictive Analytics of Individual ML Classifiers

ML Algorithma	Accuracy (95% CI)b	PPV (precision)	Recall (sensitivity)	F1 score	Kappa	Specificity	NPV	Balanced Accuracy	Ranking
Logistic regression (Lasso selection)	0.742 (0.741, 0.744)	0.583 (0.579, 0.590)	0.140 (0.140, 0.143)	0.225 (0.222, 0.227)	0.136 (0.131, 0.136)	0.963	0.753	0.551 (0.550, 0.552)	5
Support vector classifier	0.973 (0.972, 0.973)	0.972 (0.971, 0.973)	0.927 (0.925, 0.928)	0.948 (0.947, 0.949)	0.930 (0.928, 0.931)	0.990	0.973	0.956 (0.956, 0.957)	4
Random forest classifier	0.976 (0.976, 0.977)	0.999 (0.999, 0.999)	0.913 (0.912, 0.915)	0.954 (0.953, 0.955)	0.939 (0.938, 0.940)	0.999	0.969	0.959 (0.958, 0.959)	3
Extreme gradient boosting (XGBoost)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999	0.999	0.999 (0.999, 0.999)	1
Classification and regression trees	0.987 (0.987, 0.987)	0.999 (0.999, 0.999)	0.953 (0.952, 0.954)	0.975 (0.975, 0.976)	0.967 (0.967, 0.968)	0.999	0.983	0.976 (0.975, 0.976)	2
Ensemble	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999	0.999	0.999 (0.999, 0.999)	1
Artificial neural networks (deep learning)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999 (0.999, 0.999)	0.999	0.999	0.999 (0.999, 0.999)	1

Notes: CI = confidence interval; NPV = negative predictive value. ML = machine learning; PPV = positive predictive value.

^aRespective pooled threshold metrics for binary classification as computed on testing subsets (20% of the entire data, n = 559).

^bValues represent the performance accuracy (95% confidence interval).

Support Vector Classifier

In the Support Vector Classifier (SVC) models, accuracy was estimated at 97.3% (95% CI: 97.2–97.3%) using the entire data set. The accuracy was notably less in the bootstrapped with replacement (77.8%) and without replacement datasets (70.8%) (Table 4). The following hyperparameters were associated with optimal results in the SVC model: a radial basis function kernel, a scale gamma distribution, and c = 15.

Table 4.

Bootstrapping With/Without Replacement Predictive Analytics of Individual ML Classifiers

ML algorithma	Accuracy (95% CI)b	PPV (Precision)	Recall (Sensitivity)	F1 score	Kappa	Specificity	NPV	Balanced accuracy	Ranking
Logistic regression (Lasso selection)	0.736 ± 0.00	0.492 ± 0.00	0.192 ± 0.00	0.276 ± 0.00	0.152 ± 0.00	0.929 ± 0.00	0.764 ± 0.00	0.561 ± 0.00	7
	0.743 ± 0.00	0.488 ± 0.00	0.162 ± 0.00	0.243 ± 0.00	0.133 ± 0.00	0.941 ± 0.00	0.766 ± 0.00	0.552 ± 0.00	6
Support vector classifier	0.778 ± 0.00	0.556 ± 0.00	0.760 ± 0.00	0.642 ± 0.00	0.488 ± 0.00	0.785 ± 0.00	0.902 ± 0.00	0.772 ± 0.00	5
	0.708 ± 0.00	0.457 ± 0.00	0.770 ± 0.00	0.574 ± 0.00	0.373 ± 0.00	0.687 ± 0.00	0.897 ± 0.00	0.729 ± 0.00	7
Random forest classifier	0.950 ± 0.00	0.950 ± 0.00	0.861 ± 0.00	0.903 ± 0.00	0.869 ± 0.00	0.983 ± 0.00	0.950 ± 0.00	0.922 ± 0.00	1
	0.769 ± 0.00	0.658 ± 0.00	0.200 ± 0.00	0.307 ± 0.00	0.213 ± 0.00	0.964 ± 0.00	0.779 ± 0.00	0.582 ± 0.00	3
Extreme gradient boosting (XGBoost)	0.878 ± 0.00	0.964 ± 0.00	0.554 ± 0.00	0.704 ± 0.00	0.634 ± 0.00	0.992 ± 0.00	0.862 ± 0.00	0.773 ± 0.00	2
	0.796 ± 0.00	0.869 ± 0.00	0.236 ± 0.00	0.371 ± 0.00	0.294 ± 0.00	0.987 ± 0.00	0.790 ± 0.00	0.611 ± 0.00	2
Classification and regression trees	0.828 ± 0.00	0.867 ± 0.00	0.404 ± 0.00	0.551 ± 0.00	0.461 ± 0.00	0.978 ± 0.00	0.822 ± 0.00	0.691 ± 0.00	4
	0.767 ± 0.00	0.553 ± 0.00	0.458 ± 0.00	0.501 ± 0.00	0.351 ± 0.00	0.873 ± 0.00	0.825 ± 0.00	0.666 ± 0.00	4
Ensemble	0.870 ± 0.00	0.911 ± 0.00	0.560 ± 0.00	0.694 ± 0.00	0.618 ± 0.00	0.980 ± 0.00	0.863 ± 0.00	0.770 ± 0.00	3
	0.798 ± 0.00	0.720 ± 0.00	0.342 ± 0.00	0.463 ± 0.00	0.358 ± 0.00	0.954 ± 0.00	0.809 ± 0.00	0.648 ± 0.00	1
Artificial neural networks (deep learning)	0.777 ± 0.00	0.594 ± 0.00	0.465 ± 0.00	0.522 ± 0.00	0.379 ± 0.00	0.887 ± 0.00	0.824 ± 0.00	0.676 ± 0.00	6
	0.746 ± 0.00	0.513 ± 0.00	0.094 ± 0.00	0.159 ± 0.00	0.087 ± 0.00	0.969 ± 0.00	0.757 ± 0.00	0.531 ± 0.00	5

Notes: CI = confidence interval; ML = machine learning; NPV = negative predictive value; PPV = positive predictive value.

^aRespective pooled threshold metrics for binary classification as computed on testing subsets (20% of the entire data, n = 559).

^bValues represent the performance accuracy (95% confidence interval).

^cPositive predictive value.

^dNegative predictive value.

Classification and Regression Trees

The results from the CART model revealed an accuracy of 98.7% (95% CI: 98.7%–98.7%) in the whole data, compared with 82.8% and 76.7% in the bootstrap with replacement and without replacement approaches, respectively.

Random Forests Parameter Tuning

A model performance with a maximum depth = 10 and a minimum sample split = 10 was sufficient to stabilize the out-of-bag error rate for all imputed datasets. The final model in the three versions of analysis methods achieved a predictive accuracy of >97.6% (95% CI: 97.6%-97.7%) with an F1 score of 95.4% in the whole data set; 95.0% accuracy with an F1 score decreases to 95.0% the bootstrapped with replacement; and 76.9% accuracy with an F1 score decrease to 30.7% in the bootstrapped without replacement.

XGBoost, Hyperparameter Tuning, and Model Performance

After extensive grid searches, the optimal model hyperparameters for our data set were defined as a number of estimators = 300, learning rate = 0.1, maximum tree depth = 4, and gamma = 0. Running an XGBoost model on the entire training data set returned >99.9% (95% CI: 1.0, 1.0) with an F1 score of >99.9% in the whole data set; 87.8% accuracy with an F1 score decrease to 70.4% in the bootstrapped with replacement; and 79.6% accuracy with an F1 score increase to 37.1% in the bootstrapped without replacement approach.

ENSEMBLE Model Performance

We combined the outputs of logistic regression (LR), classification and regression trees (CART), support vector machine (SVM), random forest classifier (RFC), and XGBoost, capturing the unique perspectives of these base models on the data, capturing different patterns and nuances. The stacked ensemble model is an average of these individual models’ predictions, leading to a more balanced and reliable outcome by mitigating the biases of any single model, resulting in improved generalization to new data. Most predictive models exhibit high recall and specificity rates but suffer from low positive predictive value (PPV). The best classifier by individual F1 score for all three data sets was an XGBoost algorithm provided with the above hyperparameters. The XGBoost model’s predictions of unfavorable outcomes (nonstatus) were reliable, with negative predictive values of 86.2% and 79.9% for the bootstrap with replacement and without replacement approaches, respectively. When predicting positive outcomes (progression to preclinical AD), the model demonstrated high accuracy, with PPV of 99.9%, 96.4%, and 86.9% for the whole sample, bootstrap with replacement and bootstrap without replacement approaches, respectively. The model’s sensitivity in correctly classifying participants who progressed to preclinical AD was 99.9%, 55.4%, and 23.6% for the whole sample, bootstrap with replacement and bootstrap without replacement approaches, respectively. To further enhance performance, a stacked ensemble model was developed by averaging predictions from all individual models within each of the three approaches. The ensemble model, particularly using the bootstrap without replacement approach, demonstrated the best performance, improving accuracy, sensitivity, F1-score, and NPV while accounting for intra-individual correlations in participants’ longitudinal data.

Deep Learning Model

The designed deep neural network (DNN) architecture featured an input layer with fully connected neurons, each linked to every feature in the input data. The Rectified Linear Unit activation function was employed to introduce nonlinearity, enhancing the model’s capacity to capture complex patterns within the data. The input layer comprised 32 units, corresponding to the number of neurons. Additionally, two hidden layers were sequentially integrated into the model architecture, each containing fully connected neurons with a rectified linear unit activation function. These hidden layers facilitated the learning of hierarchical representations of the input data. The final layer of the DNN architecture, the output layer, generated predictions based on the learned representations from the preceding layers. This layer achieved an accuracy of 100% in the whole data set and within the 95% CI, compared with 77.0% and 74.0% in the bootstrap with replacement and without replacement approaches, respectively.

Discussion

Our study leveraged ML algorithms to classify preclinical AD using sociodemographic and genetic risk factors, cognitive assessments, neighborhood deprivation of home location, and frequently visited locations derived from daily data captured from the participant’s vehicle. When competing with multiple classifiers on this data set, our investigation found that the XGBoost algorithm provided the most robust performance in accurately classifying preclinical AD. The XGBoost algorithm, with specified hyperparameters, achieved an accuracy of 99.9% in the entire data set and demonstrated robustness even in two bootstrapped models. Additionally, the ensemble model demonstrated superior predictive accuracy based on the rigorous bootstrapping without replacement model based on the 10 predictors, suggesting that combining predictions from multiple models can effectively capture the complexity of preclinical AD. APOE ε4, age, education, home ADI, and two date differences were among the features ranked to be more important. These results highlight the necessity of applying advanced machine-learning techniques in gerontological research when investigating early disease detection and risk stratification.

In the linear approach, six out of the seven algorithms (sans LASSO) performed exceptionally well and had an accuracy of 97%–99%. However, the parameters in the test set may suggest that these models may be overfitting. Bootstrapping with replacement provided good to excellent accuracy (77%–99%) across the same six algorithms, where the RFC performed the best, followed by XGBoost, and then the ensemble model. In the most stringent model, bootstrapping without replacement, the stacking ensemble model, which combined predictions from various ML models, yielded the best performance metrics (80% accuracy), followed by XGBoost. Across all three models, LASSO performed the poorest in prediction, having the lowest performance metrics (e.g., accuracy, precision, and sensitivity in the linear and bootstrapping analyses). This is unsurprising given LASSO’s assumption of data having a linear relationship, normal distribution of errors, and constant variance. Naturalistic data can have a nonlinear distribution and be more random because it is human behavior collected at a larger volume and modeled to scale with the data size.

Our previous, machine learning studies have shown how driving behavior combined with sociodemographic variables (e.g., age, education, sex, race) can be used as a predictive model for preclinical AD using specific driving behaviors (e.g., radius of gyration, entropy; Bayat et al., 2021, 2023). This study expands on our previous work by excluding race and instead integrating GPS-derived location quantifying neighborhood deprivation via the ADI to provide a unique focus on structural and social determinants of health. Our analysis found that out of all predictors, the MoCA screen was estimated to be a more robust measure of cognitive function compared to the CDR and MMSE but barely made statistical significance in the exploratory model. Established risk factors like age (older), sex (female), APOE (ε4 carrier), and education (lower number of years) have been shown to predict preclinical AD in prior work; they were not significant in our models. Although neighborhood deprivation of a participant’s home and frequently visited destinations were not statistically significant in the models, nonetheless, their incorporation adds context, and home deprivation’s relationship to the outcome (preclinical AD) was second behind education. The home destination (ADI), had an important contribution to the accuracy of all models (see Supplemental Figure). This analysis builds on our prior work identifying these putative covariates in predicting preclinical AD. The study demonstrates that the complex integration of these factors, leveraging multiple ML algorithms in ensemble approaches, is informative and effectively captures the complex patterns associated with predicting preclinical AD.

The inclusion of socioeconomic and environmental variables, such as neighborhood disadvantage, in the model allows for a robust and holistic approach when predicting health outcomes. Studies have established the relationship between factors like deprivation and health (Stafford & Marmot, 2003) and that social factors can improve the accuracy of risk prediction (Chen et al., 2020). Including both the neighborhood deprivation of the home environment and the unique destinations participants visit via the ADI enabled a thorough model in risk prediction for assessing disease states. We demonstrate how artificial intelligence can be used to collate, analyze, and visualize longitudinal data, contributing to predictive science in gerontological science. The algorithms revealed that location and deprivation showed modest predictive value, aligning with previous research on health equity and how neighborhood, location, and socioeconomic factors can affect health.

In addition to these promising findings, this study emphasizes the necessity of interrogating the transparency of ML models in gerontological research and performing multiple cross-validations across algorithms-derivation techniques to understand their value in longitudinal data assessments, as well as the limitations associated with these methods. Transparency in ML is crucial for interpretability, reproducibility, and comparative analytics. Examining the assumptions of ML models and their requisite parameters helps to open the “black box” of algorithms and understand the underlying mechanisms driving the model predictions. This interpretable modeling style is fundamental in clinical applications, where decisions affect patient care. Interpretable models empower clinicians and researchers to determine how different variables contribute to the prediction of preclinical AD. For instance, understanding the weight and impact of sociodemographic factors, cognitive assessments, and ADI on the model’s predictions can guide targeted interventions. Transparent models also facilitate the identification of potential biases, such as those arising from socioeconomic disparities, ensuring that the ML applications do not perpetuate existing health inequities.

Limitations

Since the cohort was recruited from the greater St. Louis metropolitan area including neighboring Illinois, the findings are limited to urban settings. Most of the participants were White, well-educated, and resided in areas with lower deprivation, which limits generalizability and needs to be replicated in populations with lower education, are from lower socioeconomic backgrounds, and are more ethnoracially diverse. The reliance on naturalistic driving data necessitates that participants are active drivers, potentially excluding older adults who have ceased driving due to cognitive decline or other health issues. However, these findings can be explored by integrating additional data sources, such as wearable devices and electronic health records, to enhance predictive accuracy. Longitudinal studies with extended follow-up periods are essential to confirm early indicators of preclinical AD and their progression to prodromal AD.

Conclusions

Our study highlights the significant influence of S/SDOH on preclinical AD and demonstrates the potential of machine learning algorithms in predicting AD risk. Addressing socioeconomic disparities and leveraging advanced technologies for early detection is critical in mitigating AD risk and promoting healthy aging. As the aging population continues to grow, integrating environmental and social factors into public health strategies will be essential in addressing the challenges posed by AD and enhancing the quality of life for older adults.

Funding

This work was funded by the National Institute of Health (NIH) and National Institute on Aging (NIH/NIA) grants R01AG068183 (GMB), R01AG056466 (GMB), R01AG067428 (GMB).

Conflict of Interest

G. M. Babulal served as a co-editor for the special issue in which this article was published but was not involved in the review or decision for the article. All other authors declare no conflict of interest.

Data Availability

This study was not preregistered. Data and materials will be made available upon request. Location data (latitude, longitude) will not be provided, given this is PHI.