Mild cognitive impairment cases affect the predictive power of Alzheimer’s disease diagnostic models using routine clinical variables

Abstract

Diagnostic models using primary care routine clinical variables have been limited in their ability to identify Alzheimer’s disease (AD) patients. In this study, we sought to better understand the effect of mild cognitive impairment (MCI) on the predictive performance of AD diagnostic models. We sourced data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort. CatBoost was used to assess the utility of routine clinical variables that are accessible to primary care physicians, such as hematological and blood tests and medical history, in multiclass classification between healthy controls, MCI, and AD. Our results indicated that MCI indeed affected the predictive performance of AD diagnostic models. Of the three subgroups of MCI that we found, this finding was driven by a subgroup of MCI patients who likely have prodromal AD. This work highlights the importance of continuing to focus on better classification of the different types of MCI to improve diagnostic models of AD, rather than focusing on binary classifications between AD and control cases. Future research should focus on distinguishing MCI from prodromal AD as the utmost priority for improving translational AD diagnostic models for primary care physicians.

Introduction

Alzheimer’s disease (AD), the most prevalent dementia accounting for 50-70% of cases, is the leading cause of disability among adults over age 65¹. With a rapidly increasing prevalence, AD is expected to cost the world economy more than $14.5 trillion international dollars over the next 30 years to 2050². Improving the ability to accurately diagnose AD is of the utmost importance and ensures that patients receive appropriate support, interventions, and have the time to employ lifestyle adjustments that prolong independence and quality of life³.

Despite this, the timely and accurate diagnosis of AD remains challenging. To date, a significant number of diagnostic tools and models of AD have focused on the use of magnetic resonance imaging (MRI) and positron emission tomography (PET) scans to detect early indicators of AD pathology, including Aβ plaques^4–12. Others have relied on cerebrospinal fluid (CSF) and plasma biomarkers, including Aβ₄₂/Aβ₄₀ ratio, total tau protein, phosphorylated tau 181 (p-tau₁₈₁), p-tau₂₃₁, p-tau₂₁₇, neurofilament light (NfL), and glial fibrillary acidic protein (GFAP)^13,14. Although these tests show promise for AD diagnostics, there are many practical limitations that prevent widespread clinical implementation. Both neuroimaging scans and biomarker assays are associated with a high cost both to the healthcare system and patient, low availability, and high wait times, especially for those patients in rural areas^14–17. Further, collecting CSF is an invasive procedure that requires specific technical medical expertise¹⁴. In line with these challenges, a recent Alzheimer’s Association Primary Care Physician Dementia Care Training Survey found that half of primary care physicians do not feel that they have the local specialist resources to meet patient demand¹⁸. Although we note that this is a US-based survey and may not be generalizable globally. In fact, primary care physicians remain better able to identify those without AD than those with it (sensitivity 58%; specificity 89%)^19,20. For many patients with AD, primary care physicians are the first point of contact with the healthcare system making them essential for patient triage, diagnosis, and management¹⁵. Therefore, ensuring that primary care physicians have the skills and tools required for AD diagnostics is critical.

To improve diagnostic capabilities among primary care physicians, previous studies have used multifactorial algorithms to examine the predictive potential of routine, easy-to-obtain clinical measures. For example, these models include the Cardiovascular Risk Factors, Aging, and Dementia (CAIDE)²¹, Study on Aging, Cognition and Dementia (AgeCoDe)²², Australian National University Alzheimer’s Disease Risk Index (ANU-ADRI)²³, Rapid Assessment of Dementia Risk (RADaR) for older adults²⁴, and Brief Dementia Screening Indicator (BDSI)²⁵. There are limitations to these previous diagnostic models, however, as they report low sensitivities and positive predictive values (PPV), indicating that they are unable to reliably identify someone with AD. This was confirmed by a recent study showing existing multifactorial prediction models of AD miss 84-91% of incident AD cases and therefore have limited clinical utility²⁶.

The factors underlying the inability of these models to reliably diagnose AD remains unclear. One possibility is the presence of patients with mild cognitive impairment (MCI), which affects 10-15% of the population over age 65²⁷. Although AD first manifests clinically as MCI, not all patients with MCI will go on to develop AD^27–29. Further, MCI is known to be characterized by heterogeneous patients with multiple etiologies associated with differing clinical presentations²⁸. In line with this, there is evidence that MCI cases themselves are difficult to predict. Previous studies have reported precision, recall, and sensitivity metrics at or near chance levels for MCI classification^30,31, distinguishing MCI from AD³², and predicting conversion from MCI to AD^33–36. This suggests that these models would have low practical utility in the clinic and are likely to mislabel most MCI cases. Combined, this highlights that MCI cases may affect the predictive power of translational AD diagnostic models.

Leveraging data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort, we sought to better understand the effects of MCI cases on the predictive performance of AD diagnostic models using routine, defined as easy-to-obtain, and primary care-feasible clinical variables.

Results

CatBoost can identify healthy controls, but not MCI and AD cases

Using baseline diagnosis, we identified patients in the ADNI cohort with AD (N = 181), MCI (N = 473), and healthy controls (N = 220). There were no differences across the groups with respect to age or sex distributions, with all groups having more males than females (Supplementary Table 1). We included 120 features of routine, easy-to-obtain clinical variables (Supplementary Table 1). Using CatBoost, we first performed a sensitivity analysis using demographics, medical history, and basic primary care measures, including blood pressure and heart rate. We conducted a sensitivity analysis in which the multiclass classifier was re-fit across progressively richer feature sets chosen to reflect information typically available along the clinical pathway. Feature set 1 included demographics, medical history, and vital signs; feature set 2 additionally incorporated basic laboratory measures; feature set 3 further added neuropsychological test scores; and feature set 4 additionally included APOE genotype. Discrimination using the routine primary-care tiers (feature sets 1-2) was modest, particularly for MCI and AD (class-wise AUCs ~0.55–0.60), whereas adding neuropsychological measures and APOE genotyping substantially improved performance across all classes (Supplementary Table 2). Based on this, we then sought to determine whether any of the full set of features were able to differentiate between healthy controls, MCI, and AD patients. Our multiclass classification model was able to successfully identify healthy controls, showing high performance metrics ≥ 0.83 (Table 1). Performance metrics for MCI identification were lower, however, at ≥ 0.69 (Table 1). In the case of healthy controls, performance metrics suggested that the models were largely unable to successfully differentiate AD cases, with a low sensitivity of 0.77 and PPV of 0.57 (Table 1).

Table 1.

Performance metrics of a CatBoost model for diagnosing healthy controls, MCI, and AD using 120 clinical variables as features

	Sensitivity	Specificity	PPV	NPV	AUC
Healthy control	0.91 (0.90, 0.93)	0.94 (0.93, 0.94)	0.83 (0.81, 0.85)	0.97 (0.97, 0.97)	0.98 (0.98, 0.98)
MCI patients	0.69 (0.67, 0.70)	0.85 (0.84, 0.86)	0.84 (0.83, 0.85)	0.70 (0.69, 0.71)	0.86 (0.85, 0.96)
AD patients	0.77 (0.75, 0.79)	0.85 (0.84, 0.85)	0.57 (0.55, 0.58)	0.93 (0.93, 0.94)	0.89 (0.89, 0.90)

Data presented as mean (95% CI).

To better understand which features were being used by our CatBoost model to predict between the groups, we performed a SHAP analysis. The five most important features for identifying healthy controls included the Logical Memory-Delayed Recall, item 10 of the Geriatric Depression Scale that asks about memory problems, years of education, Logical Memory-Immediate Recall, and total score on the Geriatric Depression Scale (Fig. 1A, B). For MCI cases, the total score on the Geriatric Depression Scale was replaced by eosinophils (Fig. 1C, D) and by psychiatric medical history for AD (Fig. 1E, F).

Fig. 1. Performance of a CatBoost model and feature importance for the diagnosis of healthy control, MCI, and AD.

A Confusion matrix showing the true (correct) and false (incorrect) predictions of healthy control, MCI, and AD. B Absolute SHAP values and C heat map of the contribution of the main features for predicting healthy controls. D Absolute SHAP values and E heat map of the contribution of the main features for predicting MCI. F Absolute SHAP values and G heat map of the contribution of the main features for predicting AD. Abbreviations: BAT126 vitamin B12, BMI body mass index, GDMEMORY item 10 of Geriatric Depression Scale, GDTOTAL total score Geriatric Depression Scale, HMT3 red blood cell count, HMT4 mean corpuscular volume, HMT7 white blood cell count, HMT11 eosinophils, HMT15 neutrophils, HMT100 mean corpuscular hemoglobin, LDELTOTAL total number of story units recalled on Logical Memory-Delayed Recall, LIMMTOTAL total number of story units recalled on Logical Memory-Immediate Recall, MHPSYCH psychiatric medical history, PTEDUCAT education, RCT11 serum glucose, RCT19 triglycerides, RCT1407 alkaline phosphatase, RCT1408 lactate dehydrogenase, VSBPSYS seated systolic blood pressure, VSPULSE seated pulse rate (per minute), VSRESP respirations (per minute).

Feature selection does not improve the predictive performance of CatBoost for identifying MCI and AD patients

To identify if we could improve the performance of the CatBoost model, we performed filter-based feature selection to include only those variables significantly associated with the outcome (diagnosis). This resulted in 19 features being included (Supplementary Table 3).

As before, our model successfully identified healthy controls, although PPV increased slightly (Table 2). For MCI cases, sensitivity was slightly better relative to the full 120-feature model (Table 2). For AD cases, filter-based feature selection did not widely affect performance metrics except for a slight decrease in specificity. Across healthy controls, MCI, and AD, a SHAP analysis indicated that the most important features for predicting all three groups were Logical Memory-Delayed Recall, Logical Memory-Immediate Recall, and years of education (Fig. 2A–F). For healthy controls, additional features included memory-related item 10 and the total score of the Geriatric Depression Scale (Fig. 2A, B). For MCI, item 10 of the Geriatric Depression Scale was also important, and mean corpuscular hemoglobin (Fig. 2C, D). The two additional features that were important for predicting AD patients were psychiatric medical history and percent neutrophils (Fig. 2E, F).

Table 2.

Performance metrics of a CatBoost model for diagnosing healthy controls, MCI, and AD following feature selection

	Sensitivity	Specificity	PPV	NPV	AUC
Healthy control	0.91 (0.89, 0.92)	0.94 (0.94, 0.95)	0.85 (0.83, 0.86)	0.96 (0.96, 0.97)	0.98 (0.97, 0.98)
MCI patients	0.70 (0.68, 0.71)	0.85 (0.83, 0.86)	0.84 (0.83, 0.85)	0.70 (0.69, 0.71)	0.86 (0.85, 0.87)
AD patients	0.77 (0.74, 0.79)	0.84 (0.83, 0.85)	0.57 (0.56, 0.58)	0.93 (0.93, 0.94)	0.89 (0.88, 0.90)

Data presented as mean (95% CI).

Fig. 2. Performance of the CatBoost model and feature importance for the diagnosis of healthy control, MCI, and AD after filter-based feature selection.

A Confusion matrix showing the true (correct) and false (incorrect) predictions of healthy control, MCI, and AD. B Absolute SHAP values and C heat map of the contribution of the main features for predicting healthy controls. D Absolute SHAP values and E heat map of the contribution of the main features for predicting MCI. F Absolute SHAP values and G heat map of the contribution of the main features for predicting AD. Abbreviations: apoe apolipoprotein E genotype, GDBORED item 4 of Geriatric Depression Scale, GDDROP item 2 of Geriatric Depression Scale, GDHOPE item 14 of Geriatric Depression Scale, GDMEMORY item 10 of Geriatric Depression Scale, HMT8 neutrophils, HMT15 percent neutrophils, HMT16 lymphocytes, LDELTOTAL total number of story units recalled on Logical Memory-Delayed Recall, LIMMTOTAL total number of story units recalled on Logical Memory-Immediate Recall, MHPSYCH psychiatric medical history, NXGAIT gait on neurological exam, PTEDUCAT education, PTMARRY marital status, VSPULSE seated pulse rate (per minute).

Given that MCI is a made up of a highly heterogenous group of patients^27–29, we hypothesized that MCI patients may be affecting the predictive performance of AD diagnostic models. To identify if this was the case, we first tested the ability of CatBoost to distinguish between only healthy controls and AD patients using 19 features previously identified using filter-based feature selection. Reducing our model to a simple binary classification resulted in the ability to readily distinguish between healthy control and AD patients as indicated by high performance metrics (≥0.98; Table 3).

Table 3.

Performance metrics of CatBoost for diagnosing healthy control or AD patients

	Sensitivity	Specificity	PPV	NPV	AUC
All features	0.98 (0.97, 0.98)	0.99 (0.99, 0.99)	0.99 (0.99, 0.99)	0.98 (0.98, 0.98)	0.99 (0.99, 0.99)
Selected features	0.98 (0.97, 0.98)	0.99 (0.99, 0.99)	0.99 (0.99, 0.99)	0.98 (0.98, 0.98)	0.99 (0.99, 0.99)

Data presented as mean (95% CI).

We next sought to understand why MCI patients were affecting the predictive performance of our diagnostic models. Using a principal component analysis (PCA) based on our 19 identified features, we found that while there was clear group separation between healthy control and AD patients, MCI patients were distributed across both clusters (Fig. 3A). We identified that the MCI group could be divided into three distinct subclusters: one that overlapped healthy controls (MCI-Healthy; n = 106), one that significantly overlapped with AD patients (MCI-AD; n = 155), and a third that was distinct yet more closely related to AD (MCI-MCI; n = 212; Fig. 3B). This was further confirmed by a hierarchical cluster dendrogram (Fig. 3C) and inertia plot (Fig. 3D). To assess clinical utility, we next treated the MCI-AD subgroup as a risk-stratification target and tested whether it could be flagged using increasingly feasible feature tiers aligned with routine care. Discrimination improved in a stepwise manner as more information was added. Using demographics and routine clinical history alone provided moderate predictive performance (mean AUC = 0.70; 95% CI 0.68–0.73). Adding routine laboratory panels improved performance (mean AUC = 0.75; 95% CI 0.70–0.79). Including neuropsychological and symptom measures yielded high discrimination (mean AUC = 0.96; 95% CI 0.95–0.97). This pattern supports a staged triage approach in which routine, primary care-feasible variables enrich baseline risk for MCI-AD, with additional assessment used to increase certainty when available.

Fig. 3. Identification of the distinct subgroups of MCI patients.

A Principal component analysis showing clear separation between healthy controls and AD patients, but significant overlap of MCI. B Principal component analysis showing that MCI forms three distinct subclusters (MCI-Healthy, MCI-AD, and MCI-MCI) that have varying amounts of overlap with healthy control and AD. C Hierarchical cluster dendrogram showing that MCI-Healthy is closely related to healthy control, MCI-AD is closely related to AD, and that MCI-MCI is distinct but more related to AD. D Elbow plot showing the relationship between the number of clusters and the within-cluster sum of squared distances (inertia). The dotted line represents the optimal number of clusters.

We then looked to characterize the three subgroups of MCI using the top significantly different features (see Supplementary Table 4 for statistical analyses). This showed that the three MCI subgroups differed on apolipoprotein E ε4 (APOE4) genotype (Fig. 4A), lymphocyte count (Fig. 4B), neutrophils (Fig. 4C, D), seated pulse rate (Fig. 4E), Geriatric Depression Scale item 10 on memory complaints (Fig. 4F), and their total scores on the Logical Memory Immediate Recall (Fig. 4G) and Delayed Recall (Fig. 4H) tests. We found that MCI-Healthy was the most distinct group across these features relative to MCI-MCI and MCI-AD. The MCI-Healthy Group had a lower number of people with at least one APOE4 allele, neutrophils, and the percentage of people who replied yes to having memory complaints on the Geriatric Depression Scale. They also had higher lymphocytes and total scores on the Logical Memory tests.

Fig. 4. Comparison between the top significantly (p < 0.001) different variables between the three MCI groups (MCI-Healthy, MCI-MCI, and MCI-AD).

A Percent of patients with at least one APOE4 allele. B Lymphocyte count in blood. C Neutrophil count in blood. D Percent of neutrophils in blood. E Seated pulse rate per minute. F Percent of patients who replied yes to Geriatric Depression Scale item 10 (Do you feel you have more problems with memory than most?). G Total score on the Logical Memory Delayed Recall test. H Total score on the Logical Memory Immediate Recall test.

There were also differences across the three MCI subgroups in how stable their cognitive diagnosis was or whether it changed over time. The MCI-Healthy subgroup was the most likely to maintain a stable MCI state (Fig. 5A) or revert to being cognitively healthy (Fig. 5B) over time. The MCI–MCI subgroup was slightly more likely than the MCI–AD subgroup to maintain MCI (Fig. 5A) or revert to healthy (Fig. 5B). The MCI–Healthy subgroup, relative to both the MCI–MCI (X² = 13.294, p = 0.0039) and MCI-AD groups (X² = 36.483, p < 0.0001) was also less likely to progress to AD. There were no significant differences in the rate of progression to AD between MCI–MCI and MCI–AD subgroups (Fig. 5C).

Fig. 5. Percent of MCI patients with cognitive diagnosis stability or change over time across the three subgroups: MCI–Healthy, MCI–MCI, and MCI–AD.

A Percent of MCI patients with a stable diagnosis of MCI over time. B Percent of MCI patients who revert to cognitively healthy status over time. C Percent of MCI patients who progress to AD over time.

Finally, we sought to identify which, if any, of the three MCI subgroups was driving poor predictive performance. To do this, we implemented three CatBoost models where one of the three MCI subgroups was removed and determined the models’ ability to identify healthy controls, MCI, and AD patients. When we removed either the MCI-Healthy or MCI-MCI groups, our CatBoost models were able to identify healthy controls, as before, but were still unable to identify AD patients (Table 4). When we removed the MCI-AD subgroup, however, our CatBoost model was able to identify all patient types (Table 4), suggesting that the MCI-AD subgroup, specifically, drives poor predictive performance of our diagnostic models.

Table 4.

Performance metrics of CatBoost for diagnosing healthy control, MCI, or AD patients after removing specific MCI subgroups

	Sensitivity	Specificity	PPV	NPV	AUC
MCI–healthy group removed
Healthy control	0.78	0.99	0.96	0.95	0.97
MCI patients	0.83	0.66	0.76	0.75	0.79
AD patients	0.56	0.88	0.59	0.87	0.84
MCI–MCI group removed
Healthy control	0.92	0.97	0.92	0.97	0.99
MCI patients	0.77	0.78	0.76	0.79	0.85
AD patients	0.64	0.88	0.66	0.87	0.86
MCI–AD group removed
Healthy control	0.93	0.89	0.86	0.95	0.96
MCI patients	0.72	0.92	0.81	0.87	0.91
AD patients	0.90	0.96	0.90	0.96	0.98

Discussion

There is an urgent need to better support primary care physicians in their clinical decision-making on MCI and AD. To do this, we leveraged data from the ADNI cohort and used machine learning to determine the diagnostic potential of 120 easy-to-obtain clinical measures for MCI and AD.

Using all 120 measures as features, we found that while our model could readily identify healthy control cases, it was unable to identify MCI and AD patients and had a high level of confusion between these two diagnostic categories. Using filter-based feature selection, we narrowed down the measures to 19 that were highly correlated with the outcome. Although this led to a degree of improvement in the ability to diagnose MCI and AD sensitivities were still low, ranging between 0.71 to 0.74. This suggests that our model would likely miss around 25% of MCI and AD cases. Of note, however, was that our model outperformed existing ones^{21–25,30–32}, and we therefore sought to identify the features that were most important for diagnostic prediction. Using SHAP analysis, we showed that important features included Logical Memory Delayed and Immediate Recall scores, years of education, responses to item 10 (memory), total score on the Geriatric Depression Scale, eosinophils, and psychiatric medical history.

These findings are in line with previous research. Higher Geriatric Depression Scale scores are associated with faster cognitive decline and increased risk of MCI and AD^37–41. In line with this, AD and MCI patients have been shown to have increased incidence of psychiatric issues, including depressive, apathy, and anxiety disorders^42–44. Low levels of education are also known to be associated with an increased risk of MCI and AD^45,46. Peripheral neutrophil activation and lymphocytes, as well as a neutrophil-to-lymphocyte ratio, have been widely implicated in MCI and AD^47–52. Lower levels of hemoglobin in blood have also been linked to decreased cognitive function and AD^52,53. Although many studies show the importance of Logical Memory Immediate and Delayed Recall test scores for identifying healthy controls, MCI, and AD patients^54,55, others have indicated that they have a limited diagnostic accuracy⁵⁶. The reasons for these different findings are not clear and warrant further research.

An important finding of our study was that our models were confusing MCI and AD cases, specifically. To better understand why this was the case, we showed that MCI could be divided into three distinct subgroups: a subgroup that overlapped with healthy controls (MCI-Healthy), a subgroup that overlapped with AD (MCI–AD), and a subgroup that fell in between but was more closely related to AD (MCI-MCI). When we characterized these three subgroups, we found that they differed on key measures, including APOE genotype, lymphocytes and neutrophils, seated pulse rate, and memory as measured by item 10 of the Geriatric Depression Scale and the Logical Memory Delayed and Immediate Recall tests. To date, there has been little consensus in the literature about how many subtypes of MCI exist⁵⁷. In our study, we showed that in the ADNI cohort, there are three distinct subgroups. Other studies, however, report between two and four to five subtypes ranging from amnesic to non-amnesic MCI^28,58–61. The discrepancies may lie between using a priori definitions of MCI, largely based on the number and type of cognitive domains that are impaired, versus our approach of data-driven post hoc definitions. Although a complete comparative assessment across both approaches was outside of the scope of our current study, and the lack of sufficiently powered available data, future research would benefit from examining the merits and pitfalls of both. We also note that although APOE genotyping has not historically been part of routine clinical care, this is changing with the introduction of anti-amyloid-β monoclonal antibodies, including lecanemab and donanemab. These therapies carry a risk of amyloid-related imaging abnormalities (ARIA), including vasogenic edema (ARIA-E) and hemorrhagic changes (ARIA-H), with a higher incidence in APOE4 carriers, particularly homozygotes, prompting more restrictive eligibility and intensified MRI monitoring in this group^62,63. As a result, APOE genetic testing is now being performed routinely in clinical practice to support risk stratification, patient selection, and informed consent, and is increasingly incorporated into prescribing pathways prior to treatment initiation.

We also found significant differences in the long-term trajectories of patients within each of the three MCI subgroups. MCI-AD patients were less likely to maintain a stable MCI diagnosis or to revert to being cognitively healthy relative to the MCI-Healthy and MCI-MCI subgroups. They were also more likely to progress to AD over time. Our MCI-AD group appears to have overlap with amnesic MCI previously reported in the literature. For example, studies show that patients with amnesic MCI are more likely to progress to AD over time^64–66, overlapping with our findings here for MCI-AD. Risk of amnesic MCI also increases in patients with at least one copy of the APOE ε4 genetic variant⁶⁰, in line with our finding that the MCI–AD subgroup has the highest percentage of patients with this APOE genotype. Further, MCI–AD may be indicative of prodromal AD, as impaired delayed recall is reported to be the earliest cognitive change in this group²⁹.

Interestingly, we showed that the MCI–AD group, specifically, largely drove our models’ low ability to distinguish between MCI and AD. This finding has important implications because it suggests that only one subtype of MCI leads to poor performance of diagnostic AD models. We acknowledge that newer frameworks define AD biologically (e.g., A/T/N^67,68). However, the objective of this work is to support first-line clinical decision making using variables typically available at initial presentation, in settings where biomarker confirmation, including through PET, CSF assays, and/or specialist workup, is not consistently accessible at scale. In this study, MCI–AD denotes a subgroup within the baseline clinically defined MCI category that clusters with the AD group based on routine clinical features. Although this subgroup is enriched for expected AD risk characteristics, longitudinal trajectories within the ADNI follow-up window are heterogeneous, with many individuals remaining clinically stable with MCI. Given the absence of baseline biomarker data in this cohort, this subgroup is best regarded as clinically AD-like, rather than unequivocally AD by biological definition. Based on the data presented here, it is not clear how to get around this issue in practice. One solution is to further characterize these three subtypes of MCI and develop models that can distinguish between them as well as AD and healthy control cases. To do this, however, there is a need to prioritize the establishment of substantially larger, well-defined (i.e., many clinical measures obtained) cohorts of patients with MCI and to follow their trajectories over time. This is an especially important consideration in the context of supporting primary care physicians’ ability to undertake diagnostics. Primary care physicians have particular difficulty in correctly identifying MCI cases in their patients, and this is exacerbated by inadequate infrastructure, resources, and equipment^15,69,70. Current diagnostic methods for MCI largely rely on clinical judgment, including subjective or objective cognitive impairment^28,29,71 with a preservation of basic daily functioning^28,29, which distinguishes it from AD. However, in practice, there appear to be limitations to clinical judgment. A recent study of Medicare data from the US representing >54,000 practices and >226,000 primary care physicians showed that only 0.1% of physicians and practices have MCI diagnosis rates within the expected range⁷². Overall, our work highlights the importance of continuing to focus on differentiating MCI cases from prodromal AD to improve the translatability of effective diagnostic models of AD that can be used by primary care physicians in the clinic.

There are some additional considerations with respect to our findings and implementing them in practice. First, it is important to address the clinical relevance of distinguishing MCI from AD in primary care. While it could be argued that all individuals presenting at primary care clinics should be referred for specialist cognitive evaluation, this is often not a practical solution. Access to clinical neuropsychology services is limited in many regions, particularly in rural and remote areas, where there may be long wait times and a shortage of specialists. Additionally, referrals may be influenced by patients’ insurance status or ability to pay out-of-pocket for cognitive testing, potentially delaying diagnosis and management. In these contexts, primary care physicians play a critical gatekeeping role, and improved diagnostic tools that can differentiate MCI from early AD at the point of care are essential. Tools like this can aid in risk stratification, appropriate triaging, and early intervention, ultimately improving patient outcomes and optimizing the use of specialist resources. Therefore, developing tools that can more precisely identify prodromal AD cases is of substantial practical importance for real-world clinical benefit. A second consideration is whether primary care physicians are motivated to diagnose MCI, as some previous studies have shown that there is a low motivation due to a perceived lack of benefits for the patients⁷⁰. This is likely driven by a lack of effective treatments that physicians can offer their patients. It is still important, however, to acknowledge that providing an MCI, or even an MCI subtype, diagnosis, primary care physicians can provide patients and their families with assurances that what they are experiencing has a name and give them the knowledge required to plan for the future⁷³. Additionally, primary care physicians have reported feeling that they do not have sufficient time during a brief consultation to perform broad cognitive assessments⁷⁰. Some also feel that they lack the neuropsychological training needed to complete this type of testing, which may limit widespread translation into the clinic^15,71. Our results suggest that rather than needing to learn many complex neuropsychological questionnaires, primary care physicians only need to use the Logical Memory Immediate and Delayed Recall tests and the Geriatric Depression Scale. A limitation of our work is that we did not examine the diagnostic predictive capabilities of any AD biomarkers. For example, previous studies have linked MCI subtypes with changes in cerebrospinal fluid (CSF) total tau⁵⁹ and phospho-tau181⁷⁴. The exclusion of this data, however, was deliberate on our part as biomarkers from plasma and CSF are still not widely used, nor are they recommended for use, in clinical practice especially in the context of MCI^15,75. Further, primary care physicians lack the specialized equipment and expertise to routinely collect CSF samples from patients, and AD biomarker panels remain expensive. Despite this, as AD biomarkers increase in popularity and become more widely available, future work would benefit from examining the diagnostic potential of these in the context of differentiating between subtypes of MCI, AD, and healthy controls.

In conclusion, we have identified that a particular subgroup of MCI affects the predictive performance of AD diagnostic models using primary care routine clinical variables. This subgroup (MCI-AD) was characteristically the most similar to AD and, in line with this, was significantly more likely to progress to AD over time relative to the other MCI subgroups. These findings suggest that MCI-AD cases are likely representative of patients with prodromal AD. This work highlights the importance of AD diagnostic models, focusing specifically on differentiating MCI cases from prodromal AD cases (who are also diagnosed as MCI) as the main way to improve their diagnostic predictive power and translatability.

Methods

Data and patients from the ADNI cohort

This retrospective study used data generated from the ADNI cohort, publicly available at https://ida.loni.usc.edu/, using data downloaded in April 2024. Patients in the ADNI cohort were diagnosed as either cognitively healthy, MCI, or AD based on the presence of subjective memory complaints, a Mini-Mental State Exam (MMSE) score, and Clinical Dementia Rating (CDR)⁷⁶. Healthy controls had no subjective memory complaints, MMSE range of 24-30, and CDR of 0. Patients with MCI had subjective memory complaints, MMSE 24-30, and CDR of 0.5. AD patients similarly had subjective memory complaints, a lower MMSE of 20–26, and a CDR of 0.5 to 1.0⁷⁶. Supplementary Table 1 shows the demographic characteristics of this cohort. We included continuous (e.g., weight, height, body mass index (BMI), and routine blood tests) and categorical (e.g., apolipoprotein E (APOE) genotype, and physical exam) variables obtained at patients’ baseline visits that we determined would be appropriate for a primary care physician to complete (Supplementary Table 1). We also generated a set of cardiovascular features to better capture physiological dynamics potentially associated with MCI and AD. Some of these, including augmented pulse pressure $\left(\mathrm{systolic}\mathrm{BP}-\mathrm{diastolic}\mathrm{BP}\right)*\mathrm{pulse}$⁷⁷ and mean arterial pressure $\mathrm{diastolic}\mathrm{BP}+\frac{1}{3}*(\mathrm{systolic}\mathrm{BP}-\mathrm{diastolic}\mathrm{BP})$⁷⁸$,$ have been used previously and are established clinical indices. Other derived features, including pulse to pressure ratio $\frac{\mathrm{pulse}}{(\mathrm{systolic}\mathrm{BP}-\mathrm{diastolic}\mathrm{BP})}$, heart stress index $\frac{\mathrm{pulse}*(\mathrm{diastolic}\mathrm{BP}+\frac{1}{3}*\left(\mathrm{systolic}\mathrm{BP}-\mathrm{diastolic}\mathrm{BP}\right))}{\mathrm{systolic}\mathrm{BP}}$, and systolic blood pressure to diastolic blood pressure ratio $\frac{\mathrm{systolic}\mathrm{BP}}{\mathrm{diastolic}\mathrm{BP}}$ are not standard clinical metrices. These were included as exploratory engineered variables designed to capture potential non-linear or composite interactions between heart rate and blood pressure components that may not be reflected by raw measures alone. This study only used publicly accessible, de-identified data from the ADNI cohort and therefore did not require independent ethics approval. The ADNI cohort study was approved by the institutional review boards of the participating ADNI centers, all patients provided informed consent, and the study was completed in accordance with the Declaration of Helsinki. We used the STROBE cohort checklist when writing our report⁷⁹.

Statistical analyses and machine learning

In our initial experiments, all 120 clinical variables (Supplementary Table 1) were used as features in our multi-class diagnostic model. We also performed a filter-based feature selection method to determine the clinical variables that were highly associated with the outcome (i.e., diagnosis). Here, we used a Kruskal–Wallis test (p < 0.01) for identifying significant continuous features and Cramer’s V (>0.11) for categorical ones.

To evaluate the diagnostic potential of the clinical variables, we used CatBoost⁸⁰, a gradient boosting framework optimized for both categorical and continuous variables. The complete dataset was initially split randomly into an 80% training set and a 20% held-out testing set to evaluate the final models’ performance. To address class imbalances across healthy control, MCI, and AD patients, we performed oversampling using the “imblearn” library⁸¹, where the underrepresented class was randomly resampled. During model development, the training set was further randomly subdivided into training and validation datasets using stratified k-fold cross-validation with four folds to ensure balanced representation across classes and mitigate potential biases. Hyperparameter fine-tuning was performed using the “optuna” library⁸², which employs a Bayesian optimization approach to efficiently search for the optimal combination of parameters. These included the following, with ranges in brackets: maximum number of trees that can be built (iterations; 100–2000), learning rate (0.01–0.3), coefficient at the L2 regularization term of the cost function (1–10), Bayesian bootstrap parameter (0.5–10.0), randomness for scoring splits (1.0–3.0), depth of the trees (3–10), minimum number of training samples in a leaf (1–100), and percentage of features to use at each split selection (0.5–1.0).

To identify the clustering across healthy controls, MCI, and AD patients, we first performed a principal component analysis (PCA) using the filter-based selected features. We then identified subclusters (subgroups) of MCI patients by taking principal component (PC) 1 and PC2 and using Gaussian mixture models for clustering. We confirmed these results using a hierarchical cluster dendrogram. To characterize heterogeneity within clinical MCI as a risk-stratification problem, we defined an AD-proximal MCI subgroup based on proximity to AD versus control in a low-dimensional latent space. Within each outer cross-validation split, we fit a PCA on the outer-training set only, computed AD and control centroids, and assigned each MCI participant an AD-proximity score (distance-to-control minus distance-to-AD). Training fold quantiles were then used to define MCI subgroups, and the same thresholds were applied to the held-out fold (fold-safe assignment). We then trained parsimonious classifiers within the clinical MCI group to predict MCI subgroups using three pre-specified feature tiers: demographics/clinical history, plus routine laboratory measures, and plus neuropsychological/symptom measures. All preprocessing and model fitting occurred within training folds, and performance was summarized by AUC with 95% CIs across outer evaluations. MCI subtype removal experiments were conducted to visualize how MCI heterogeneity affects model behavior. Because this analysis is intended to demonstrate failure modes rather than provide definitive performance estimates, we report results from a single 70% train, 30% unseen test split. All experiments and optimizations were conducted using Python (v.3.11.7) with the libraries listed above and “pandas”, “numpy”, “matplotlib”, “seaborn”, “sklearn”, “catboost”, and “scipy”, and Google Colab’s GPU-accelerated environment, which facilitated faster model training and evaluation. Source code is available at https://github.com/Art83/adni_mci.

Model evaluation and leakage control

Model performance was evaluated using a fully nested cross-validation framework to obtain unbiased generalization estimates and prevent data leakage, with all preprocessing confined to training data. An outer cross-validation loop was used to estimate out-of-sample performance, and an inner loop was used for model selection and hyperparameter tuning. Within each outer split, all steps, including feature filtering/selection, imputation, scaling, and hyperparameter optimization, were fit using the outer-training fold only. Hyperparameters were selected within the outer-training fold via inner cross-validation using random search, after which the final model was refit on the full outer-training data and evaluated once on the held-out outer-test fold. To reduce dependence on any single partition, the outer procedure was repeated across multiple randomizations, yielding 50 outer-fold evaluations. We report mean performance with 95% confidence intervals computed from the empirical distribution of outer-fold scores. For binary comparisons, we report sensitivity (correctly identified positive cases), specificity (correctly identified negative controls), positive predictive value (PPV; true positives/all predicted positives), negative predictive value (NPV; true negatives/all predicted negatives), and AUC. For multi-class models, we report balanced accuracy and macro-averaged AUC (one-vs-rest), together with class-wise metrics derived from out-of-fold predictions. We note that while AUC is a commonly used performance metric in machine learning studies, it provides only limited insight into clinically relevant performance⁸³. This is supported by prior AD prediction models that report high AUCs but low sensitivity, limiting their ability to identify incident AD cases²⁶. Accordingly, we emphasize threshold-dependent metrics (sensitivity/specificity and PPV/NPV) alongside AUC. Feature importance was assessed using SHAP (Shapley Additive exPlanations) from one evaluation to quantify each feature’s contribution to model predictions⁸⁴. This allowed us to identify those specific features that are most likely to act as translational diagnostic variables for primary care physicians. SHAP was done in Python (v.3.11.7) using the “shap” library.

Author contributions

C.A.F. and A.S. jointly contributed to the concept and design, interpretation of data, and critical review of the paper for important intellectual content. A.S. performed the statistical analyses. C.A.F. acquired the data and wrote the paper. All authors reviewed the paper.

Data availability

The data used in this study are from the ADNI cohort and are available at https://ida.loni.usc.edu/.

Code availability

Source code is available on GitHub at https://github.com/Art83/adni_mci.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41514-026-00390-w.