Comprehensive evaluation of AT(N) imaging biomarkers for predicting cognitive decline

Abstract

INTRODUCTION

Few studies have investigated how different Alzheimer's disease biomarker definitions affect predictions of cognitive decline.

METHODS

We predicted cognitive decline using 42 biomarker definitions spanning amyloid beta, tau, and neurodegeneration in 383 ADNI participants with baseline neuroimaging and longitudinal cognitive assessments. We identified optimal predictors for each pathology and variable type (continuous, binary, non‐binary categorical) using nested cross‐validation. Models were compared against support vector machines (SVMs) incorporating brain‐wide pathology. In a subsample, we compared imaging‐based biomarkers with biofluids (cerebrospinal fluid and plasma).

RESULTS

We observed substantial variability in accuracy, even for measures of the same pathology. Tau biomarkers were most accurate, performing comparably to models incorporating all three pathologies. Binarization of biomarkers mostly preserved accuracy, while SVMs trained on brain‐wide pathology data generally did not improve performance. Imaging‐based biomarkers consistently outperformed biofluid ones.

DISCUSSION

Imaging‐based biomarker definitions are not interchangeable and can influence the assessment of cognitive decline.

Highlights

• Comparison of many (n = 42) AD biomarkers for predicting cognitive decline.

• Tau biomarkers outperform measures of amyloid or neurodegeneration.

• Binarization of biomarkers mostly does not limit prediction accuracy.

1. BACKGROUND

The biological definition of Alzheimer's disease (AD) relies on biomarkers that can accurately quantify AD pathobiological disease processes, such as the aggregation of amyloid beta (Aβ) plaques and tau neurofibrillary tangles. ¹ , ² , ³ Importantly, biomarkers enable AD detection prior to symptomatic onset. Because of their capabilities, biomarkers have been used in a variety of research settings including disease classification, ⁴ cognitive forecasting, ⁵ subtype identification, ⁶ clinical trial stratification, ⁷ disease staging, ⁸ , ⁹ and more. Moreover, biomarkers are becoming increasingly important for the clinical management of AD. ² For instance, recently approved anti‐Aβ treatments for AD require biomarker evidence of Aβ pathology.

AD‐sensitive biomarkers vary in terms of modality, underlying pathology, and statistical formulation. Idiosyncrasies in approaches used to define biomarkers may result in unwanted variability when applied for clinical and research uses. For example, cut points for positron emission tomography (PET) and cerebrospinal fluid (CSF) biomarker binarization are application specific, ¹⁰ , ¹¹ , ¹² and alternative thresholding approaches may result in considerable variability for group assignment. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ Little is known about how variability in biomarker definitions affects prediction of cognition in AD.

Identifying which biomarkers are most predictive of cognitive trajectories can inform our biological understanding of AD. Moreover, precise cognitive decline predictions are valuable for identifying candidates for early therapeutic interventions and for establishing meaningful cognitive endpoints in clinical trials. Despite these implications, investigations into the ramifications of different biomarker operationalizations remain limited. One previous study found that different biomarker definitions varied in their ability to predict longitudinal Mini‐Mental State Examination (MMSE) scores and that binarization reduced the predictive power of some biomarkers relative to continuous values. ¹³ Similar analyses in separate cohorts with additional cognitive measures are needed to confirm and extend these findings, particularly to establish optimal biomarker combinations for both prognostic accuracy and mechanistic insight.

Here, we compiled a comprehensive set of neuroimaging measures (42 in total) covering the AD core biomarkers Aβ (A) and tau (T), as well as non‐specific biomarkers of neurodegeneration (N). We systematically evaluated how biomarker categories and individual variants differ in their ability to predict longitudinal cognitive decline. We additionally applied machine learning to test how traditional biomarker approaches compare to methods which can detect more complex, multivariate patterns in imaging data. Finally, we tested multiple CSF and plasma biomarkers and compared their performance with imaging‐based measures.

2. METHODS

2.1. Participants

Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). The ADNI was launched in 2003 as a public‐private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), PET, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early AD. For up‐to‐date information, see www.adni‐info.org. All data used in this study are accessible from ADNI following formal data usage agreements.

We included ADNI participants (n = 383) with baseline T1‐weighted MRI, Aβ‐PET ([¹⁸F]‐Florbetapir), tau‐PET ([¹⁸F]‐Flortaucipir), and longitudinal cognitive assessments. Exclusion criteria were gaps between scans longer than 1 year and missing values for age, sex, APOE genotype, Clinical Dementia Rating ® (CDR®), ¹⁷ MMSE, ¹⁸ and Phenotype Harmonization Consortium (PHC) cognitive composites. ¹⁹

2.2. Image acquisition and processing

MRI and PET derivatives were generated by the ADNI PET Core. Freesurfer (version 7.1.1) was applied to MRI scans to generate volumes within 68 cortical ²⁰ and 14 subcortical ²¹ gray matter regions of interest (ROIs). PET standardized uptake value ratios (SUVRs) were generated for these ROIs after co‐registration of PET to MRI. Further details are provided in Supplementary Material.

2.3. Cognitive and clinical assessments

Longitudinal change in MMSE scores was the primary dependent variable for this study. LMEMs were used to model longitudinal change in MMSE scores and define individualized measures of MMSE change over time (ΔMMSE). LMEMs were fit with random slopes and intercepts for subjects. To assess whether LMEM uncertainty was related to downstream prediction error, bootstrapping (1000 resamples) was used to estimate individualized standard errors. We also assessed whether the variable length of cognitive follow‐up affected results by repeating the calculation of ΔMMSE in a subsample (n = 229) where we restricted follow‐up length to 2 years (requiring each individual to have a MMSE assessment at 2 years ± 6 months). We also predicted longitudinal change in CDR sum of boxes (ΔCDR‐SB), modified Preclinical Alzheimer's Cognitive Composite scores (ΔPACC), ²² and neuropsychological composites defined by PHC ¹⁹ for memory (ΔMEM), executive functioning (ΔEXF), and language (ΔLAN). Details of PACC calculation are provided in the Supplementary Methods. CDR was used as a measure of dementia severity, with subjects labeled as either cognitively unimpaired (CU, CDR = 0) or cognitively impaired (CI, CDR ≥ 0.5).

2.4. Imaging‐based biomarker definitions

We evaluated a variety of imaging‐based biomarker definitions (full list in Table S1). Biomarkers were categorized by pathology (AT[N]) and variable type (binary [BIN], non‐binary categorical [CAT], continuous [CON]). Atlas regions used to form composite measures are listed in Table S2.

Continuous variables consisted of scalar MRI (volume) and PET (SUVR) measures. Continuous measures of Aβ included the mean cortical SUVR ²³ , ²⁴ (Aβ composite) and Centiloid. ²⁵ Continuous tau measures included average uptake in meta‐temporal regions (MT) ¹¹ and in Braak stage regions ²⁶ , ²⁷ (Braak I, Braak III/IV, Braak V/VI). Braak II was omitted due to off‐target binding issues with flortaucipir. ²⁸ , ²⁹ Hippocampal volume and volume of the MT region were included as continuous assessments of neurodegeneration.

Binary predictors consisted of dichotomized versions of the aforementioned continuous predictors. Three methods were tested for binarization: previously published cutoffs, Z‐scoring, and Gaussian mixture modeling (GMM). Previously published SUVR cutoffs were included for Aβ (1.11, ³⁰ 1.24, ³¹ 1.30, ¹¹ 1.42 ¹¹ ) and tau ¹¹ (1.20, 1.21, 1.23, 1.33). We also tested Centiloid cutoffs (15, 20, 25, 30) based on ranges reported in previous literature. ³² , ³³ , ³⁴ Z‐scoring and GMMs were applied to the Aβ mean cortical SUVR, MT tau SUVR, MT volume, and hippocampal volume. Z‐scores for each variable were computed relative to CU, Aβ‐negative individuals. Z‐scores were dichotomized using cutoffs of 2 and 2.5 standard deviations. GMM binarization was implemented by fitting two‐component Gaussian mixtures to the distribution of continuous variables. The cutoff point was defined as the intersection between the fitted Gaussians. GMMs were not applied for hippocampal volume, due to the absence of bimodal distribution.

RESEARCH IN CONTEXT

1. Systematic review: PubMed and Google Scholar were used to find papers which compare different biomarker definitions for AD. Additional studies were found by investigating cited sources, where relevant.

2. Interpretation: Our results revealed that different operationalizations of the same pathology (i.e., amyloid, tau, neurodegeneration) offered varying predictive value for modeling cognitive decline. Tau biomarkers were the most accurate. Binary versions of biomarkers generally did not perform worse than non‐binary alternatives. Imaging measures consistently outperformed cerebrospinal fluid (CSF) and plasma analytes.

3. Future directions: Our approach can be extended to include other datasets, additional imaging modalities, and biomarkers of additional pathologies. We only compared biomarker definitions in the context of predicting cognitive decline, but other applications can also be investigated (e.g., clinical trial stratification, predicting cognitive conversion).

Non‐binary categorical biomarkers included quartiles, binarization with an indeterminate zone, and staging systems. Quartiles were computed by binning continuous values at the 25th, 50th, and 75th percentiles. Binarization with an intermediate zone (BIZ) was used to model the uncertainty of assigning individuals with biomarker values near the cutoff threshold. ² BIZ was implemented with a GMM, where individuals were marked as uncertain if they showed less than 60% probability of being assigned to either Gaussian component. Staging systems were included as non‐binary categorical measures that assign disease severity grades based on the spatial extent of Aβ or tau pathology. For Aβ, we applied two previously published staging models. ⁹ , ³⁵ For tau, we implemented two versions of Braak staging with different granularities: a five‐stage scheme (I, III, IV, V, VI) and a three‐stage scheme (I, III/IV, V/VI). Detailed description of the staging procedures for each of these systems is provided in the Supplementary Methods.

2.5. Fluid‐based biomarkers

We identified a subsample of individuals who underwent CSF immunoassays within 1 year of imaging (n = 214). CSF samples were analyzed with Roche Elecsys kits targeting Aβ₄₂, Aβ₄₀, total‐tau (t‐tau), and tau phosphorylated at threonine 181 (p‐tau181). CSF concentrations of analytes were grouped into biomarker categories as previously recommended ¹ (Aβ: Aβ₄₂, Aβ₄₀, Aβ₄₂/Aβ₄₀, tau: p‐tau181, neurodegeneration: t‐tau). Raw concentrations were included as continuous measures. GMMs were used to define binary versions of CSF biomarkers. BIZ and quartiles were used to generate non‐binary categorical versions.

We selected a separate subsample of individuals who had readouts for plasma biomarkers (n = 278). We selected biomarkers for Aβ (Aβ₄₂, Aβ₄₀, Aβ₄₂/Aβ₄₀), tau (p‐tau217), and neurodegeneration (neurofilament light chain [NfL]). NfL concentrations were generated with Quanterix assays, while other plasma markers were generated with Fujirebio. Like our CSF analysis, we generated binary (GMM) and non‐binary categorical (BIZ, quartiles) versions of the continuous plasma concentrations. Additionally, we applied the recommended cutoffs from the ADNI documentation for Aβ₄₂/Aβ₄₀ (negative > 0.1053; positive < 0.0820; intermediate 0.0821 to 0.1052) and pTau217 (negative < 0.0055; positive > 0.0086; intermediate 0.0056 to 0.0085) to define BIZ biomarkers. We also used the midpoint of the two BIZ cutoffs as the threshold for binarizing Aβ₄₂/Aβ₄₀ and p‐tau217.

2.6. Statistical analyses

We conducted a series of cross‐validated (implementation details described in Supplementary Methods) modeling experiments to compare biomarkers for predicting cognitive outcomes. We first ran separate models for each biomarker to assess the predictive value of each included definition. Training data were used to fit linear regressions where ΔMMSE was predicted from a single biomarker and covariates (age, sex, APOE E4 positivity). Biomarker definitions with tunable parameters (Z‐scores, GMMs) were also fit with training data. In each cross‐validation iteration, a baseline model was also trained using only covariates as predictors. Trained models were then evaluated on the corresponding testing fold.

We next developed models including multiple biomarkers to test whether combinations of biomarkers improved prediction accuracy and whether models incorporating continuous or non‐categorical binary biomarkers outperformed those using binary biomarkers. These analyses used nested cross‐validation with 10 outer folds and five inner folds. Within the inner cross‐validation loop, a model selection procedure identified the best performing individual predictors. Specifically, models were grouped based on the pathology (AT[N]) and variable type (binary/non‐binary categorical/continuous), and the best performing definition within each group was selected based on the lowest mean root mean squared error (RMSE) across the five test folds. The outer cross‐validation was then used to train new linear models combining the biomarkers selected in the inner cross‐validation (see Table S3 for all combination models).

Finally, we used support vector machine (SVM) regression to directly predict ΔMMSE from imaging data. SVMs were trained using regional Aβ uptakes, tau uptakes, or gray matter volumes as input features. An additional model combined all these regional imaging features into a multimodal predictive model. SVM training used a nested cross‐validation design with 10 outer folds and five inner folds. The inner loop was used for hyperparameter tuning (Supplementary Methods.).

All cross‐validation experiments had 10 outer folds and were repeated 10 times, yielding 100 out‐of‐sample error estimates per model. Model error was assessed using RMSE. To statistically evaluate differences in accuracy for comparisons of interest, we used Nadeau‐Bengio t‐tests (which correct for the interdependency of cross‐validated estimates ³⁶ , ³⁷ ) with false discovery rate correction. ³⁸ Feature importance was assessed by examining the magnitude of regression coefficients for linear regression models and the covariance‐corrected weights for SVM models ³⁹ (see Supplementary Methods.).

Finally, we conducted additional cross‐validated experiments to (1) predict change in CDR‐SB, PACC, or neuropsychological domains instead of ΔMMSE (i.e., ΔCDR‐SB, ΔPACC, ΔMEM, ΔEXF, and ΔLAN), (2) restrict testing to CU or CI individuals, and (3) compare CSF‐ or plasma‐based biomarkers to image‐based ones.

3. RESULTS

Our sample consisted of 383 (CU: n = 223, CI: n = 160; Table 1) individuals with baseline biomarker imaging and longitudinal cognitive assessments (mean [SD] years = 3.32 [1.47], mean [SD] visits = 2.91 [0.98]). Length of cognitive follow‐up was longer for CU individuals (mean [SD] years = 3.72 [1.35], mean [SD] visits = 2.83 [0.85]) than CI individuals (mean [SD] years = 2.77 [1.46], mean [SD] visits = 3.01 [1.15]).

TABLE 1.

Sample characteristics.

Variable	CU (n = 223)	CI (n = 160)	p
Age	73.76 (7.07)	76.13 (8.37)	0.003 **
Sex			0.007 **
Female	127 (57.0)	68 (42.5)
Male	96 (43.0)	92 (57.5)
Race			0.082
White	192 (86.1)	150 (93.8)
Black	22 (9.9)	4 (2.5)
Asian	4 (1.8)	2 (1.2)
Other	4 (1.8)	3 (1.9)
Unknown	1 (0.4)	1 (0.6)
Ethnicity			0.962
Not Hispanic	209 (93.7)	151 (94.4)
Hispanic	14 (6.3)	9 (5.6)
Education	16.67 (2.42)	15.99 (2.72)	0.011 *
APOE ε4			0.854
Carrier	79 (35.4)	59 (36.9)
Non‐carrier	144 (64.6)	101 (63.1)
Centiloid	19.92 (35.65)	46.23 (54.82)	<0.001 ***
Meta‐temporal tau (SUVR)	1.19 (0.11)	1.40 (0.38)	<0.001 ***
Hippocampal volume	5.47 (0.61)	4.83 (0.73)	<0.001 ***
Global CDR			<0.001 ***
0.0	223 (100.0)	0 (0.0)
0.5	0 (0.0)	130 (81.2)
1.0+	0 (0.0)	30 (18.8)
MMSE	29.07 (1.22)	26.62 (3.53)	<0.001 ***
ΔMMSE	−0.11 (0.22)	−0.81 (0.92)	<0.001 ***
CDR‐SB	0.04 (0.15)	2.39 (2.31)	<0.001 ***
ΔCDR‐SB	0.06 (0.18)	0.69 (0.89)	<0.001 ***
PACC	0.01 (0.73)	−1.83 (1.64)	<0.001 ***
ΔPACC	−0.05 (0.09)	−0.29 (0.24)	<0.001 ***
No. of cognitive visits	2.83 (0.85)	3.01 (1.15)	0.082
Length of follow‐up (years)	3.72 (1.35)	2.77 (1.46)	<0.001 ***

Note: The last column reports p values for significance tests comparing distributions of variables across dementia status groups (*p<0.05, **p<0.01, ***p<0.001). Chi‐squared tests were used for categorical variables (sex, race, ethnicity, APOE status, global CDR) and one‐way ANOVAs were used for continuous variables (age, Centiloid, tau SUVR, hippocampal volume, MMSE, ΔMMSE, CDR‐SB, ΔCDR‐SB, PACC, ΔPACC, number of visits, length of follow‐up).

Abbreviations: CDR‐SB, Clinical Dementia Rating Sum of Boxes; MMSE, Mini‐Mental State Examination; PACC, Preclinical Alzheimer's Cognitive Composite; SUVR, standardized uptake value ratio.

3.1. Prediction accuracy for individual biomarkers

Relative to the baseline covariate‐only model (RMSE = 0.686 ± 0.15, mean ± SD), almost all biomarkers significantly improved the prediction of ΔMMSE (Figure 1). The only exceptions were meta‐temporal volume and hippocampal volume binarized at −2.5 Z‐scores (both p > 0.05). Gains in accuracy varied across biomarkers (range in RMSE reduction: 3.8% to 26.8%). Tau biomarkers were most predictive, with the 10 most accurate biomarkers being tau‐based. Furthermore, SVM models trained on regional pathology exhibited higher accuracy than individual biomarkers. The tau SVM was the best performing model overall (RMSE = 0.502 ± 0.12, p < 0.001), while the Aβ SVM (RMSE = 0.609 ± 0.16, p < 0.05) and volume SVM (RMSE = 0.623 ± 0.13, p < 0.05) were the best performing Aβ and neurodegeneration models, respectively. Among non‐SVM approaches, the best performing models for each AT(N) category were Centiloid (RMSE = 0.631 ± 0.14, p < 0.01), three‐stage Braak (RMSE = 0.522 ± 0.12, p < 0.001), and MT volume (RMSE = 0.622 ± 0.12, p < 0.01).

FIGURE 1.

Boxplots showing performance of individual biomarkers for predicting ΔMMSE. Values represent prediction errors (RMSE) from out‐of‐sample predictions across 100 cross‐validation instances (lower values indicate higher accuracy). The baseline model included only covariates as independent variables (mean performance indicated by the dotted line). All other models included the same covariates and a single amyloid (maroon), tau (green), or neurodegeneration (blue) biomarker. Models are sorted first by biomarker and then by accuracy. Labels on the top axis indicate the biomarker variable type (BIN = binary, CAT = non‐binary categorical, CON = continuous, SVM = support vector machine).

3.2. Prediction accuracy for combined AT(N) biomarkers

We next applied a model selection routine to compare categories of biomarkers based on AT(N) pathology and variable type (continuous, binary, non‐binary categorical) (Figure 2, Figure S1). Relative to the baseline covariate‐only model (RMSE = 0.686 ± 0.15), all categories of biomarkers resulted in improvements in prediction accuracy (all p < 0.05), except for binary neurodegeneration (N_BIN ^, t = 1.61, p > 0.05). Improvements in accuracy were greatest for continuous tau predictors (T_CON, t = 5.11, p < 0.001) and smallest for categorical Aβ predictors (A_CAT, t = 1.83, p < 0.05). Combining biomarkers across the three AT(N) pathology categories further improved performance, but gains were primarily driven by the inclusion of tau. Models combined AT(N) predictors were more accurate than Aβ‐only (BIN: t = 3.16, p < 0.01; CAT: t = 3.03, p < 0.05; CON: t = 4.16, p < 0.001) and neurodegeneration‐only models (BIN: t = 4.11, p < 0.001; CAT: t = 3.39, p < 0.01; CON: t = 4.14, p < 0.001), but not tau‐only ones (BIN: t = 1.33, p > 0.05; CAT: t = 0.03, p > 0.05; CON: t = 1.60, p > 0.05). A similar pattern was observed for SVMs. Radial basis function (RBF) kernels were consistently selected over linear kernels (Figure S2). The combined AT(N) SVM (RMSE = 0.506 ± 0.010) outperformed the Aβ (A_SVM, t = 2.74, p < 0.05) and neurodegeneration (N_SVM, t = 4.24, p < 0.001) SVMs, but not the tau version (N_SVM, t = 0.22, p > 0.05). Across models, prediction residuals were minimally explained by error in LMEM fit (R ² = 0.015 to 0.058) or length of cognitive follow‐up (R ² = 0.029 to 0.078). We additionally tested whether including continuous or non‐binary categorical predictors could improve performance over binarized versions (Figure 3). Relative to the binary‐only AT(N) model (A_BIN/T_BIN/N_BIN: RMSE = 0.542 ± 0.010), we found no significant improvements in prediction accuracy when including non‐binary measures of any pathology (all p > 0.05). SVM models also did not outperform the binary‐only model (all p > 0.05).

FIGURE 2.

Boxplots showing predictive performance for ΔMMSE across biomarker types. Cross‐validation was used to identify the best performing biomarkers grouped by pathology (AT[N]) and variable type (BIN = binary, CAT = non‐binary categorical, CON = continuous, SVM = support vector machine). Models are compared against a baseline using only covariates (median performance indicated by the dotted line). Colors are used to indicate the variable type of included biomarkers (yellow: binary, purple: non‐binary categorical, red: continuous, blue: SVM). Lighter shading indicates single pathology models, while darker coloring indicates models with Aβ, tau, and neurodegeneration biomarkers. Gold stars indicate a significant improvement in accuracy relative to the baseline. Gray stars and bars highlight significant pairwise differences between models. Statistical significance was assessed using Nadeau‐Bengio t‐tests with correction for multiple comparisons (*p < 0.05, **p < 0.01, ***p < 0.001).

FIGURE 3.

Boxplots showing predictive performance for ΔMMSE when binary biomarkers are compared with alternative definitions. The baseline model (yellow) included only binary AT(N) measures (median performance indicated by dotted line). Subsequent models replaced binary predictors with non‐binary categorical (purple), continuous (red), or support vector machine (SVM) alternatives. Lighter coloring indicates models in which one AT(N) biomarker was replaced, while darker coloring indicates all three AT(N) dimensions were replaced. No significant differences were found between the models (all p > 0.05).

Results were unchanged when prediction performance was evaluated using Spearman correlation instead of RMSE (Figure S3). We also found similar results when predicting ΔMMSE calculated with a maximum follow‐up length of 2 years (n = 229), although effect sizes were smaller (Figure S4A). Most biomarkers improved prediction accuracy, except for binary amyloid (A_BIN), binary neurodegeneration (N_BIN), non‐binary categorical neurodegeneration (N_CAT), and continuous neurodegeneration (N_CON). Tau‐only models performed as well as combined AT(N) models, and biomarker combination was primarily beneficial over neurodegeneration‐only models. We similarly found no evidence of benefit for inclusion of non‐binary biomarkers (Figure S4B).

3.3. Feature importance analysis

Inspection of model coefficients highlighted the importance of tau for predicting cognition. For regression models that included multiple biomarkers as predictors, coefficients were highest for tau, followed by neurodegeneration and Aβ (Figure 4A). When considering continuous biomarkers in particular, weights for Aβ were much lower than those of tau or neurodegeneration. Similarly, the weights for tau features were on average higher than those of Aβ or atrophy in the combined AT(N) SVM model (Figure 4B). Cortical weights for tau and neurodegeneration highlighted medial and lateral temporal structures, while Aβ weights were more evenly distributed across the cortex. Subcortical regions were weighted lower than cortical ones, except for tau uptake and gray matter volume in the amygdala and hippocampus.

FIGURE 4.

Feature importance analysis. (a) Coefficients for Aβ, tau, and neurodegeneration biomarkers from cross‐validated linear regression models. Values are taken from 100 instances of the all‐binary (A_BIN/T_BIN/N_BIN) and all‐continuous (A_CON/T_CON/N_CON) models. (b) Brain maps showing average regional feature importance derived from the A_SVM/T_SVM/N_SVM model.

3.4. Comparison of prediction accuracy in CU and CI populations

We observed similar results when restricting predictions to CI participants (Figure S5). Relative to the covariate‐only model (RMSE = 0.964 ± 0.27), all categories of biomarkers increased prediction accuracy for ΔMMSE (range in RMSE reduction: 7.4% to 26.5%, all p < 0.05). Combined AT(N) biomarkers further improved prediction accuracy over Aβ‐only and neurodegeneration‐only models (all p < 0.05). Combined AT(N) models also outperformed tau‐only models when using binary (A_BIN/T_BIN/N_BIN vs T_BIN: t = 2.74, p < 0.05) or continuous (A_CON/T_CON/N_CON vs T_CON: t = 2.69, p < 0.05) predictors. Non‐binary biomarkers generally did not improve on the binary‐only model except when incorporating continuous measures of tau (A_BIN/T_CON/N_BIN vs A_BIN/T_BIN/N_BIN: t = 2.69, p < 0.05).

Results differed, however, when restricting predictions to CU participants (Figure S6). Models with Aβ‐only and neurodegeneration‐only predictors did not outperform the baseline model (all p > 0.05), except for the Aβ‐SVM (t = 11.09, p < 0.001) and neurodegeneration‐SVM (t = 9.80, p < 0.001). Alternatively, all tau‐only models performed better than the baseline (T_BIN: t = 3.67, p < 0.001, T_CAT: t = 4.00, p < 0.001, T_CON: t = 5.92, p < 0.001), as well as combined AT(N) models (A_BIN/T_BIN/N_BIN: t = 4.32, p < 0.001; A_CAT/T_CAT/N_CAT: t = 5.03, p < 0.001; A_CON/T_CON/N_CON: t = 5.97, p < 0.001). As in prior analyses, binary‐only predictive models performed comparably to non‐binary models, except for the tau‐SVM (t = 3.35, p < 0.01) and combined AT(N) SVM (t = 3.32, p < 0.01).

3.5. Prediction accuracy using alternative cognitive assessments

Detailed reporting of experiments with alternative cognitive endpoints is presented in Supplementary Results. We observed consistent results when predicting ΔCDR‐SB (Figure S7) or ΔPACC (Figure S8). All biomarker types improved prediction accuracy (relative to covariates), but improvements were driven by tau. Only for continuous biomarkers did we see evidence of combination models outperforming tau‐only models. As previously, there was no evidence of non‐binary biomarkers outperforming binary alternatives, except for the AT(N) SVM when predicting ΔPACC. The amyloid SVM and tau SVM also surpassed continuous biomarker definitions when predicting ΔPACC.

Similar patterns were observed when predicting neuropsychological domain scores (Figures S9–S11). Results for predicting ΔMEM and ΔLAN largely mirrored that of predicting ΔMMSE: Most biomarker definitions improved prediction accuracy and tau‐only (but not amyloid‐only or neurodegeneration‐only) models performed as well as combination AT(N) models. Findings diverged for ΔEXF, with results showing more moderate effect sizes, less clear separation of tau from other biomarkers, and no benefits for SVMs. Across neuropsychological domains, non‐binary definitions performed similarly to binary versions, excepting continuous tau and continuous AT(N) models for ΔLAN.

3.6. Comparison of prediction accuracy for imaging‐based and fluid‐based models

In a subsample with CSF assay data (n = 214; Table S4), CSF‐based models showed limited ability to predict prospective cognition (Figure 5A). Models incorporating CSF‐based biomarkers did not outperform the covariate‐only baseline model (all p > 0.05). Imaging‐based models were significantly more accurate than CSF‐based models, regardless of variable type (BIN: t = 2.37, p < 0.05; CAT: t = 2.73, p < 0.01, CON: t = 3.55, p < 0.01).

FIGURE 5.

Boxplots comparing predictive accuracy for ΔMMSE using imaging‐based (solid) and fluid‐based (hatched) biomarkers. (a) Results comparing imaging‐based biomarkers to CSF‐based biomarkers. (b) Results comparing imaging‐based biomarkers to plasma‐based biomarkers. For both (A) and (B), values represent RMSEs from 100 cross‐validation runs. Colors indicate biomarker variable type (yellow: binary, purple: non‐binary categorical, red: continuous). Gold stars denote significant improvement in accuracy relative to the covariate‐only baseline model. Gray stars and connecting bars indicate significant pairwise differences between models. Statistical significance was assessed using Nadeau‐Bengio t‐tests with correction for multiple comparisons (*p < 0.05, **p < 0.01, ***p < 0.001).

In the plasma biomarker subsample (n = 278; Table S5), we found modest evidence of plasma biomarkers improving prediction of ΔMMSE but not to the level of imaging. Inclusion of binary plasma biomarkers improved prediction accuracy (t = 1.99, p < 0.05); however, non‐binary categorical and continuous versions did not (p > 0.05). For both binary and non‐binary categorical definitions, imaging biomarkers outperformed plasma (BIN: t = 2.76, p < 0.05; CAT: t = 2.56, p < 0.05).

Similar results were found when predicting ΔMMSE for CU individuals (Figure S12). No CSF biomarkers improved prediction over baseline. The same was true for plasma, although continuous plasma biomarkers approached statistical significance (t = 1.76, p = 0.06).

4. DISCUSSION

AD biomarkers differ along several dimensions, including the underlying pathology they measure (e.g., Aβ, tau), modality (e.g., imaging, CSF, blood), and measurement characteristics (e.g., variable type). Our analyses indicate that differences along these dimensions result in considerable variability when imaging‐based biomarkers are utilized in downstream tasks. Notably, even biomarkers targeting the same pathology vary in accuracy when used to model cognition. Careful selection of biomarker definitions is therefore important for predictive tasks, as certain operationalizations may be less informative than others.

While nearly all tested biomarkers improved the prediction of cognition, some categories yielded consistently larger gains. Consistent with prior work, ⁴⁰ , ⁴¹ , ⁴² multiple analyses demonstrated that tau biomarkers exhibited stronger associations with cognition than assessments of Aβ or neurodegeneration. This was further supported by feature importance analyses, which showed that tau predictors received higher weights in models incorporating all three AT(N) categories. In most experiments, models including tau performed comparably to those including all three AT(N) categories. Furthermore, this pattern was largely consistent when predicting different cognitive outcomes and neuropsychological composite scores (particularly memory and language). Together, these findings underscore the importance of tau burden in predicting future cognitive impairment. We included SVM‐based predictors to assess whether whole‐brain, multivariate patterns would outperform traditional biomarker definitions. In general, SVM models achieved higher predictive accuracy. For example, within each AT(N) category, the best performing individual predictor was the SVM model. RBF kernels were consistently selected over linear kernels during our SVM hyperparameter tuning, potentially indicating the presence of non‐linear pathology‐cognition relationships. That said, the gains in accuracy were generally not statistically significant relative to traditional biomarkers. This indicates that traditional biomarkers capture much of the relevant pathological change contributing to cognitive decline. However, model differences were evaluated using the conservative Nadeau–Bengio t‐test with correction for multiple comparisons. Consequently, the absence of statistically significant differences should not be interpreted as evidence that the models perform equivalently; rather, modest improvements may have not been detected. We opted to use SVMs as a machine learning comparison method based on the prominence of the method in AD predictive setting, ⁴ the interpretability of SVM weights, ³⁹ , ⁴³ and the relative stability of SVM accuracy across hyperparameters. ⁴⁴ Further work will be required to investigate whether our findings are consistent across other machine learning methods.

Relative to binary biomarkers, continuous or non‐binary categorical biomarker assessments did not diminish predictive performance when predicting global cognitive decline. This contrasts with prior work reporting reduced accuracy with binarized biomarkers, particularly for amyloid in CU and tau/neurodegeneration in CI populations. ¹³ These discrepancies may reflect methodological differences. For instance, we evaluated several binarization strategies and selected the best performing ones via cross‐validation, whereas the previous study focused on GMM‐based cutoffs without explicitly measuring out‐of‐sample performance. Additionally, prior analyses emphasized quantitative comparisons based on variance explained (R ²), whereas we focused on predictive accuracy (RMSE) and formally tested the statistical significance of performance differences. Given these divergent results, further work is warranted to determine whether binarization results in loss of information relevant for cognitive forecasting. SVMs also generally did not surpass binary biomarkers, indicating the incorporation of multivariate brain‐wide pathology patterns was not particularly informative for our predictive task, relative to a conventional biomarker definition. A notable exception was observed in CU individuals, where the tau and AT(N) SVMs offered gains over the binary model. This finding suggests that, in preclinical populations, predictive signal may extend beyond predefined meta‐ROIs commonly used in standard biomarker definitions.

While most imaging‐based biomarkers improved the prediction of cognitive decline, the same was generally not true of their biofluid counterparts. Models incorporating CSF biomarkers as predictors did not outperform covariate‐only baseline models, regardless of the analyte or its operationalization. The same was true of plasma, except for binary versions. Previous studies similarly demonstrated stronger associations between imaging‐based biomarkers and cognitive scores in AD relative to fluid biomarkers. ⁴⁵ Biofluid analytes primarily reflect earlier pathological processes that may plateau before the onset of neurodegeneration and cognitive decline. ² , ⁴⁶ Notably, imaging‐based biomarkers still outperformed biofluid biomarkers even when restricting predictions in CU individuals. As such, biofluid biomarkers may be less suited for modeling cognitive decline and more suited for diagnosis. However, our imaging and biofluid comparison experiments should be interpreted cautiously, as they are based on a single predictive task in a non‐random subsample, which is underpowered relative to our main analysis. Further direct comparison of biomarker modalities across diverse prediction tasks will help clarify the relative strengths of each modality.

This study has limitations that should be considered. First, while we performed a comprehensive and rigorous evaluation of multiple biomarkers, biomarkers such as fluorodeoxyglucose‐PET, cortical thickness, and functional imaging were not included and warrant further investigation. Our work also does not address markers of other pathological processes relevant to AD, for example, synaptic dysfunction, neuroinflammation, and vascular disease. Second, analyses were restricted to ADNI because the inclusion of other sources posed issues of harmonization and biomarker availability. While ADNI uniquely enables the analyses we conducted, it remains limited in demographic diversity. ⁴⁷ As such, our results warrant replication in more diverse cohorts. Third, we treated annualized change in cognition, estimated with LMEMs, as a fixed, error‐free endpoint. While we found little evidence that LMEM uncertainty meaningfully affected our results, more sophisticated designs that directly predict future cognitive scores and model uncertainty are warranted. Finally, cognitive prediction represents only one application of AD biomarkers. The relative utility of different biomarker definitions may be different for other applications, such as clinical trial stratification or treatment monitoring.

The rapid growth of large data initiatives has led to a proliferation of approaches to quantifying AD pathology. When selecting among the plethora of these approaches, it is important for researchers to consider their biological, statistical, and practical characteristics. Our results demonstrate that different operationalizations of the same pathology can yield markedly different performance in downstream predictive tasks. While combining biomarkers spanning multiple pathologies maximized accuracy for predicting cognitive decline, tau biomarkers alone (across operationalizations) appeared to drive model performance. Binary versions of biomarkers generally performed as well as non‐binary alternatives (including multivariate machine learning), indicating the former may be sufficient for modeling cognitive worsening. Finally, imaging biomarkers may offer more prognostic value for predicting cognitive decline than biofluid alternatives.

AUTHOR CONTRIBUTIONS

Tom Earnest: Conceptualization; methodology; software; validation; formal analysis; investigation; data curation; writing (original draft); writing (review & editing); visualization. Braden Yang: Methodology; software; data curation; writing (review & editing). Deydeep Kothapalli: Data curation; writing (review & editing). Aristeidis Sotiras: conceptualization; resources; writing (review & editing); supervision; project administration; funding acquisition). The Alzheimer's Disease Neuroimaging Initiative (ADNI) provided data for this study.

CONFLICT OF INTEREST STATEMENT

Author AS has equity in TheraPanacea and has received personal compensation for serving as grant reviewer for BrightFocus Foundation. The remaining authors have no conflicting interests to report. Author disclosures are available in the Supporting Information.

CONSENT STATEMENT

Informed consent was obtained from all participants in the Alzheimer's Disease Neuroimaging Initiative (ADNI). The study was approved by the Institutional Review Boards at each participating site. The authors accessed the ADNI data through the USC Laboratory of Neuro Imaging (LONI) Image & Data Archive (IDA) following the data usage agreement.

Supporting information

Supporting Information: dad270480‐sup‐0001‐SuppMat.pdf

Earnest T, Yang B, Kothapalli D, Sotiras A. Comprehensive evaluation of AT(N) imaging biomarkers for predicting cognitive decline. Alzheimer's Dement. 2026;18:e70480. 10.1002/dad2.70480