Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures

Abstract

INTRODUCTION

Digital speech biomarkers (DSBs) support the detection and monitoring of Alzheimer's disease (AD) in Latinos. However, they have not been benchmarked against standard cognitive and neuroimaging measures, missing a critical validation milestone.

METHODS

Thirty‐three AD patients and 33 healthy controls completed verbal fluency tasks, episodic memory and executive tests, and magnetic resonance imaging (MRI) (volume) and functional MRI (fMRI) (connectivity) scans. Between‐group machine learning classification was compared among fluency‐derived DSBs, episodic and executive test scores, MRI, and fMRI measures.

RESULTS

The fluency classifier's performance (area under the curve [AUC] = 0.84) was comparable (p > 0.14) to the episodic (AUC = 0.90), executive (AUC = 0.79), and structural (AUC = 0.90) classifiers and superior to the functional classifier (AUC = 0.65, p = 0.002). Top discriminating features were word length and frequency, both associated with right (pre)frontal volume upon adjusting for sociodemographic factors.

DISCUSSION

DSBs appear non‐inferior to standard cognitive and imaging measures, supporting scalable AD assessments in Latinos.

Highlights

• We examined digital speech biomarkers (DSBs) for detecting AD in Latinos.

• DSBs were benchmarked against cognitive and neuroimaging features.

• DSB‐based classifiers matched or outperformed cognitive and brain classifiers.

• Top DSBs included word length, phonological neighborhood, and frequency.

• Word length and frequency correlated with right (pre)frontal brain volume.

1. BACKGROUND

Alzheimer's disease (AD) is characterized by primary episodic memory and executive deficits, ¹ medial temporal lobe atrophy, ² aberrant default mode network (DMN) connectivity, ³ and amyloid beta (Aβ) and tau pathology. ⁴ A leading cause of disability and death, ¹ AD reduces quality of life for patients and caregivers, ¹ affecting physical, social, and mental well‐being and generating massive annual (health)care costs. ¹ Its burden is particularly severe in Latin America, ⁵ where cases multiply amid a lack of biofluid and neuroimaging devices, culturally adapted tests, and specialized clinicians. ⁶ The need has arisen for scalable markers enabling AD detection and monitoring in this population. ⁷

Prominent among these are digital speech biomarkers (DSBs) – acoustic and linguistic anomalies derived automatically from verbal production. These low‐cost, time‐efficient metrics allow for detecting AD, ⁸ , ⁹ predicting its onset preclinically, ¹⁰ , ¹¹ and differentiating it from other dementias. ¹² , ¹³ Moreover, they capture cognitive symptom severity, ¹² , ¹⁴ underlying brain atrophy patterns, ¹² , ¹⁵ and Aβ status, ¹⁵ reinforcing clinical utility. ⁷ While most evidence comes from English speakers, ¹⁴ accumulating research on Latinos ⁷ has revealed that (semi)spontaneous speech features are sensitive to AD but not to other disorders ¹³ , ¹⁶ and that some support their detection even when models are trained on English speakers. ¹⁷ Additional results come from verbal fluency tasks, with word property (WP) ¹² and word organization ¹⁸ metrics proving sensitive to AD. Promisingly, apps capturing these patterns are being deployed across Latin American healthcare settings. ¹⁹ , ²⁰

A good balance between sensitivity and interpretability is achieved by metrics tapping on semantic memory, which is typically affected by AD. ²¹ First, speech timing (ST) features – quantifying pauses and spoken segments – capture word retrieval efficiency. Individuals with or at heightened risk for AD exhibit more and longer pauses than healthy controls (HCs), ²² reaching classification accuracies of up to ∼0.85. ²³ , ²⁴ , ²⁵ , ²⁶ Second, WP features index vocabulary search patterns, ¹² , ²⁷ , ²⁸ revealing more frequent, less specific terms in AD patients than in HCs, yielding area under the receiver operating characteristic curve (AUC) scores as high as 0.89, ⁹ , ¹² , ²⁷ and tracking cognitive symptom severity. ¹² Findings from diverse populations, including Latinos, show that both ST and WP features predict temporal/frontal atrophy ¹¹ , ¹² and abnormal DMN connectivity. ¹² Importantly, these metrics can be reliably captured from verbal fluency tasks, which are brief, engaging, and widely used worldwide. Thus, they represent powerful candidate markers for Latinos. ⁷

However, ST and WP metrics lack validation against clinical and neuroimaging benchmarks, a key milestone to prevent advocating suboptimal measures and increasing inequities in Latin America. ²⁹ Classification using WP and ST features approximates that of episodic, executive, and otherwise cognitive tests (AUC = 0.80 to 0.97; accuracy = 0.77 to 0.96), ³⁰ , ³¹ , ³² as well as those from MRI features such as hippocampal volume (AUC = 0.87 to 0.99; accuracy = 0.85 to 0.99). ³³ Yet, these figures stem from different populations and methodologies, limiting comparability and interpretability. Only a few studies, on non‐Latino participants, have contrasted digital speech metrics with cognitive outcomes in the same cohort (revealing similar or better discrimination), ¹⁰ , ¹⁵ , ³⁴ , ³⁵ and none has done so against neuroimaging measures in any population, let alone targeting ST and WP metrics. Briefly, a critical validation milestone is missing for this thriving field.

Here, we benchmarked ST and WP metrics against cognitive and imaging measures in Latinos. ST and WP metrics were extracted from two 1‐min verbal fluency tasks. ³⁶ Episodic memory features were derived from immediate and delayed recall tests, while executive features came from short‐term memory, working memory, attention, and mental set‐shifting tests. Volumetric and connectivity features were extracted from MRI and fMRI recordings, respectively. Machine learning classifiers were independently trained on each feature set to discriminate between patients and HCs, and their outcomes were statistically compared. The most discriminative fluency features were further examined via brain–behavior correlations.

We hypothesized that classification via ST and WP metrics would approximate that of episodic, executive, and imaging measures. Also, we expected classification to be driven by pause‐related features, ¹¹ , ²⁴ word frequency, granularity, and phonological neighborhood. ¹² Finally, we anticipated that such top features would correlate with the volume and connectivity of temporal and frontal brain regions. ¹¹ , ¹² This approach fills a key validation gap for speech biomarkers of AD in Latinos.

RESEARCH IN CONTEXT

1. Systematic review: We reviewed studies on DSBs in AD using PubMed and Google Scholar. We considered papers in any language from Quartile 1 and 2 journals. References in each hit were then scanned for additional works. Few studies compared AD classification performance using DSBs versus cognitive metrics. None of them compared DSBs and neuroimaging measures, let alone in Latin American samples. All pertinent studies were duly referenced.

2. Interpretation: Our study suggests that AD classification through DSBs is non‐inferior to that achieved using standard measures.

3. Future directions: Replications should be conducted in larger, more diverse samples. Also, larger feature sets should be used, employing further cognitive and neuroimaging measures, and including social determinants of health and biofluid biomarkers. Finally, our approach should be tested across different languages to meet recent calls for cross‐linguistic equity in the field.

2. METHODS

The study's methods are summarized in Figure 1.

FIGURE 1.

Experimental design. (A) Data acquisition. We recruited (A1) 33 individuals with AD and 33 HCs, who completed (A2) phonemic and semantic fluency tasks, (A3) cognitive tests tapping on episodic memory (Craft story and FCSRT) and executive function (TMT‐A, TMT‐B, forward digit span, and backward digit span), and (A4) MRI and fMRI brain scans. (B) Feature extraction. (B1) Fluency features consisted of word property and speech timing metrics from each fluency task. (B2) Episodic features were derived from normalized immediate and delayed scores from the Craft story task and FCSRT. (B3) Executive features were derived from TMT‐A and B time and errors, and digit forward and backward span. (B4) Structural features consisted of AD‐implicated ROI volume derived from MRI scans. (B5) Functional features were obtained from the internal connectivity of canonical networks, derived from fMRI scans. (C) Machine learning pipeline. We trained multiple models for each feature group through nested cross‐validation (with 11 folds of inner layer and three folds of outer layer), using 10 random seeds and bootstrapping at a factor of 1000. We then evaluated performance in terms of AUC and accuracy, comparing the best classifier of each category. Finally, the most important features of the best fluency classifier were correlated with gray matter volume in a whole‐brain VBM analysis in SPM12. AD, Alzheimer's dementia; AUC, area under receiver operating characteristic curve; FCSRT, Free and Cued Selective Reminding Test; fMRI, functional magnetic resonance imaging; HC, healthy control; MRI, magnetic resonance imaging; ROI, region of interest; SPM12, statistical parametric mapping 12; TMT, trail making test; VBM, voxel‐based morphometry.

2.1. Participants

As in previous research, ¹² participants were selected from a larger N of 73 (38 AD patients, 35 HCs) through an automated algorithm that ensured balanced groups and statistical matching for sex, handedness, age, years of education, and acquisition center (Table 1). The final sample comprised 66 native Spanish speakers (33 with AD, 33 HCs), providing an adequate feature‐to‐sample ratio for machine learning analysis under the n − 1 criterion ³⁷ (Section 2.6). Participants were recruited from two centers of the Multi‐Partner Consortium to Expand Dementia Research in Latin America (ReDLat), following unified procedures that included extensive neurological and neuropsychological interviews and evaluations. ³⁸ , ³⁹ A proportion of these participants (11 of the 33 AD patients, and 13 of the 33 HCs) were part of a previous report by Ferrante et al. ¹²

TABLE 1.

Participants’ sociodemographic and cognitive profiles.

Variable	AD patients (n = 33)	HCs (n = 33)	Stats	p‐value
Sociodemographic profile
Sex (F:M) a	21:12	17:16	0.558	0.455
Handedness (L:R) a	3:30	0:33	1.397	0.237
Age, years b	72.18 (8.62)	70.18 (8.43)	0.953	0.344
Education, years b	14.12 (4.12)	15.24 (4.04)	−1.115	0.269
Acquisition center (1:2) a	12:21	16:17	0.558	0.455
Cognitive profile
MMSE c	22.45 (3.75)	29.03 (1.33)	−9.488	<0.001*
Normalized immediate recall score b	0.18 (0.16)	0.60 (0.16)	−10.987	<0.001*
Normalized delayed recall score d	0.11 (0.16)	0.65 (0.23)	34.0	<0.001*
Digits forward d	4.76 (1.39)	5.39 (1.12)	379.0	0.030*
Digits backward d	3.18 (1.01)	4.12 (1.11)	300.5	0.001*
TMT‐A time d	104.97 (54.27)	52.94 (21.75)	823.5	<0.001*
TMT‐A errors d	0.26 (0.63)	0.28 (0.72)	497.0	0.991
TMT‐B time d	409.12 (308.78)	134.59 (76.34)	893.0	<0.001*
TMT‐B errors d	1.89 (2.06)	1.00 (1.76)	605.5	0.015*

Note: Data presented as mean (SD), except for sex and handedness.

Abbreviations: AD, Alzheimer's disease; F, female; HC, healthy control; L, left‐handed; M, male; MMSE, Mini‐Mental State Examination; R, right‐handed; TMT‐A, Trail‐Making Test Part A; TMT‐B, Trail‐Making Test Part B.

^a P values calculated via chi‐squared test (χ ²).

^b P values calculated with independent samples Student's t‐test.

^c P values calculated with independent samples Welch's t‐test.

^d P values calculated with independent samples Mann‐Whitney U test.

^e Scores were normalized via the min‐max method as some participants were tested with the Craft Story Recall task and others with the Free and Cued Selective Recall Test.

In line with standardized validated protocols, ⁴⁰ AD participants were diagnosed by expert neurologists following clinical criteria from the National Institute of Neurological and Communicative Diseases and Stroke ⁴¹ as well as the Alzheimer's Disease and Related Disorders Association. ⁴² Biomarker data were unavailable for this cohort. Analysis of MRI and fMRI scans (Supporting Information 1) revealed bilateral hippocampal atrophy (Supporting Information 2) and reduced connectivity between (para)hippocampal and frontal regions as well as within the DMN (Supporting Information 3) in AD patients. They were also impaired in overall cognitive skills, as measured by the Mini‐Mental State Examination (MMSE) ⁴³ ; episodic memory, examined with the Craft story retelling task ⁴⁴ and the Free and Cued Selective Reminding Test (FCSRT) ⁴⁵ ; short‐term and working memory, assessed through standard forward and backward digit span tasks, respectively ⁴³ ; and attention and cognitive flexibility, as evaluated via the Trail‐Making Test Parts A and B (TMT‐A, TMT‐B), ⁴³ respectively – see Sections 2.3 and 2.4 for task details. HCs were recruited through an outreach program and invitations extended to eligible caregivers of patients. They were confirmed to be functionally autonomous and cognitively preserved, with MMSE scores above the local mean of 26. ⁴⁶ No participant reported other neurological disorders, psychiatric conditions, primary language deficits, or substance abuse.

All participants provided written informed consent in accordance with the Declaration of Helsinki. The study was approved by the institutional ethics committee.

2.2. Fluency tasks and features

All participants completed phonemic and semantic fluency tasks (Figure 1A2), requiring them to utter as many words as possible starting with the phoneme /p/ and fitting the category “animals,” respectively. A certified neuropsychologist administered these tasks in a counterbalanced order in a silent testing room. Following reported procedures, ¹² , ⁴³ participants had 60 s per task and were instructed to avoid proper names, numbers, repetitions, words from the same family, or morphological variations of the same word. Instructions were given orally, including examples of invalid responses.

All responses were audio‐recorded and automatically transcribed using Google's Text‐to‐Speech Application Programming Interface. Transcriptions were then checked and edited by at least two examiners following a standardized protocol. ¹² Unintelligible words, metalinguistic comments, and interventions from the evaluators were discarded. As in recent work, ¹² features were extracted from all remaining words, regardless of their validity in terms of standard scoring criteria. All participants produced at least five words per task.

2.2.1. Speech timing features

Audio recordings were preprocessed with the Toolkit to Examine Lifelike Language (TELL) app. ¹⁹ , ²⁰ They were converted to .wav format, downmixed to mono (single channel), encoded in GSM Full Rate, resampled to 16 kHz, compressed with a factor of 8, bandpass‐filtered from 200 Hz to 3.4 kHz, loudness‐normalized on the FFmpeg multimedia framework, and denoised through the FullSubNet model. Finally, voice activity was then detected to segment audio signals into speech and non‐speech regions.

Then, using a PRAAT script, ⁴⁷ we automatically extracted the following ST features from each recording: number of phonated segments (peaks in intensity preceded and followed by dips in intensity), number of pauses (silent intervals), articulation rate (number of syllables over phonation time), mean and standard deviation of phonated segment duration, and mean and standard deviation of pause duration. Therefore, each participant had seven ST features per task (Figure 1B).

2.2.2. Word properties

As in prior studies, ¹² we extracted seven lexico‐semantic properties from each of the participants’ transcribed responses. First, each word's granularity was determined using Python's library NLTK to access WordNet, a hierarchical graph of nodes representing increasingly specific concepts leading from the broadest hypernym (“entity”) to more specialized terms (e.g., “animal,” “dog,” “bulldog”). Granularity refers to the number of nodes between a word and the top‐node “entity.” For instance, bin‐3 words are closer to “entity” than bin‐10 words, with the former indicating more general concepts. As in previous research, ¹² , ¹⁶ Spanish responses were automatically translated into English for this analysis.

Additionally, we used the EsPal database ⁴⁸ to extract each response's word frequency (logarithmic frequency per million), phonological neighborhood (number of words generated by substituting, adding, or omitting a phoneme), syllabic length (number of syllables for word), phonemic length (number of phonemes in each word), and concreteness (rated on a scale from 1: not concrete to 7: highly concrete). For word concreteness, missing values were imputed by retrieving the 20 closest words using the 150‐dimensional FastText model for Spanish and subsequently selecting the concreteness value from the closest word available in the EsPal database using a Python script. We calculated the mean and standard deviation (SD) of each variable per subject. Therefore, each participant had 12 WP features per task (Figure 1B).

2.3. Episodic memory tasks and features

Episodic memory was assessed through the Craft story retelling task ⁴⁴ in 19 AD patients and 17 HCs and through the FCSRT ⁴⁵ in 14 AD patients and 16 HCs. For the Craft story task, participants were read a short narrative and asked to recall it immediately and again after a 20‐min delay, with performance quantified as the number of story units correctly recalled at each time point. ⁴⁴ For the FCSRT, participants learned words under controlled encoding with semantic cues and completed recall immediately and after a 20‐min delay, with performance quantified using total recall scores. ⁴⁵ Both tasks yielded an immediate recall score and a delayed recall score, with higher scores indicating better episodic memory performance. To render outcomes comparable across tasks, immediate and delayed recall scores were normalized between tasks via min–max normalization. Episodic classifiers were trained with two features: normalized immediate recall score and normalized delayed recall score (Figure 1B2).

2.4. Executive tasks and features

Executive assessments were conducted to evaluate four core domains affected by AD. Short‐term and working memory were assessed via forward and backward digit span tests, respectively. ⁴³ The former requires participants to recall a sequence of numbers in the same order, while the latter involves recalling the sequence in reverse order. In both tasks, sequence length increases progressively. Participants must correctly recall each sequence twice to advance, whereas two consecutive incorrect responses result in task termination. Performance is scored based on the maximum sequence length correctly recalled in two trials, with no upper limit. ⁴³ Attention and mental set shifting were evaluated with the TMT‐A and TMT‐B, two tests with good specificity and sensitivity. ⁴³ In TMT‐A, participants are asked to connect 25 numbers randomly arranged on a page in ascending order. In TMT‐B, they must alternate between 25 numbers and letters (e.g., 1‐A‐2‐B). Scoring is based on the completion time of each task, which ranges from 1 to 300 (seconds), with both tasks taking approximately 10 min to complete. ⁴³ Executive classifiers were trained using six features: the short‐term memory score, the working memory score, TMT‐A time and errors, and TMT‐B time and errors (Figure 1B3).

2.5. Neuroimaging protocol and brain features

2.5.1. MRI and fMRI data acquisition and preprocessing

MRI and fMRI recordings were performed in two scanners, with minimally varying parameters (Supporting Information 1). As in previous research, ¹² acquisition center was introduced as a covariate in all analyses.

Participants’ gray matter volume was estimated via voxel‐based morphometry, following reported protocols. ⁴⁹ Preprocessing was performed with the CAT12 toolbox for SPM12 ⁴⁹ running on MATLAB R2021a. Scans were automatically set with origins to the anterior commissure and reoriented. Images were segmented between gray and white matter, normalized to the MNI space based on a Dartel estimation, and smoothed. Total intracranial volume (TIV) was estimated using native‐space tissue maps obtained via a CAT12 module.

For fMRI preprocessing, scans were manually set with origins to the anterior commissure and reoriented, and the first five volumes were removed. ¹² Functional and anatomical data were then preprocessed using a standard preprocessing pipeline in the CONN version 22.a toolbox. ⁵⁰ This included (1) realignment with correction of susceptibility distortion interactions; (2) outlier detection using ART (global signal z‐value threshold = 9, subject‐motion mm threshold = 2); (3) direct segmentation, MNI‐space normalization and resampling to 3 mm isotropic voxels; (4) smoothing with a Gaussian kernel of 8 mm full width at half maximum; (5) denoising (regression of potential confounding effects characterized by white matter, cerebrospinal fluid, realignment, and scrubbing); and (6) bandpass filtering between 0.008 and 0.09 Hz. Quality control measures did not differ between groups (Supporting Information 1). Still, motion artifacts were regressed out during the default denoising step implemented in the CONN toolbox.

2.5.2. Structural brain features

For the structural features, based on the Automated Anatomical Labelling 3 (AAL3) atlas, ⁵¹ we estimated the volumes of regions of interest (ROIs) typically atrophied in AD. ² We used the CAT12 “ROI estimation” function to extract volumetric measures for the following ROIs: hippocampus, parahippocampal gyrus; superior, middle, and inferior temporal gyri; temporal pole (superior and middle temporal gyri); precuneus; inferior frontal gyrus (opercular and triangular parts); rolandic operculum; superior frontal gyrus (medial orbital); gyrus rectus; insula; anterior cingulate cortex (subgenual, pregenual, and supracallosal parts); middle cingulate and paracingulate gyri; and posterior cingulate gyrus. Each ROI was analyzed separately for the left and right hemispheres, yielding a total of 38 structural features, with which the structural classifiers were trained (Figure 1B4).

2.5.3. Functional features

Functional brain features were derived using ROI‐to‐ROI connectivity analysis in the CONN toolbox, based on the AAL3 atlas. ⁵¹ Fisher‐transformed bivariate Pearson correlation coefficient matrices were extracted as absolute values using custom Python 3.9 scripts with the SciPy library, capturing the strength of functional interactions irrespective of sign. ⁵² Eight resting‐state networks were examined: (1) default mode, (2) language, (3) executive control, (4) salience/ventral attention, (5) visual, (6) somatomotor, (7) limbic, and (8) memory networks (Supporting Information 3). The global correlation value was also included as an additional feature, yielding a total of nine functional features, with which the functional classifiers were trained (Figure 1B5).

2.6. Machine learning

Machine learning analyses were conducted to classify between patients and HCs based on (a) fluency features, (b) episodic memory features, (c) executive features, (d) structural brain features, and (e) functional brain features (Figure 1C). Fluency classifiers were trained using (i) ST features alone, (ii) WP features alone, and (iii) the combination of both. Episodic memory classifiers were trained with normalized immediate and delayed recall scores. Executive classifiers were trained using a combination of digit span forward and backward scores and TMT‐A and TMT‐B outcomes (completion time and errors). Structural brain classifiers were based on the volume of the aforementioned AD‐relevant ROIs. Functional brain classifiers were trained considering connectivity within each of the networks listed above. Additional structural and functional brain classifiers were run with alternative feature sets (Supporting Information 4).

For each feature set, we employed a stratified nested cross‐validation procedure, where label balance is maintained across folds. ⁵³ The outer loop used three folds for model evaluation, while the inner loop used 11 folds for hyperparameter tuning and feature selection. Following prior studies, ¹² inside the inner loop, feature values were standardized using z‐score ⁵⁴ normalization, and missing data were imputed using a k‐nearest neighbors approach (k = 5). This entire process was repeated across 10 random seeds. We tested three classification algorithms: XGBoost (an ensemble method that reduces the risk of overfitting by averaging the predictions made by several weak learners), ⁵⁵ k‐nearest neighbors, and logistic regression, to consider both linear and non‐linear decision boundaries. Hyperparameter tuning was carried out using Bayesian optimization, ⁵⁶ with 15 initial sampling points and 50 optimization iterations. Feature selection was performed via recursive feature elimination, ⁵⁷ using the drop in AUC as the criterion for feature removal. Classification performance of the optimal models was evaluated using 95% confidence intervals (CIs) computed via bootstrap resampling (N = 1000), pooling the scores across all random seeds. ⁵⁸ For each feature set, the best‐performing model was defined based on AUC. All machine learning analyses were implemented in Python 3.11.9 using the Scikit‐learn library (version 1.4.1). Custom scripts were developed to automate feature selection, hyperparameter tuning, and model evaluation. The full pipeline was executed on a high‐performance computing cluster, enabling parallel processing to optimize model training and evaluation efficiency.

Model performance was evaluated using multiple metrics, including AUC, accuracy, F1 score, sensitivity, precision, normalized expected cost, ⁵⁹ and normalized cross‐entropy, ⁶⁰ following recent recommendations. ⁶¹ Feature importance was assessed for each top classifier after training them on the entire dataset. Finally, we compared the best‐performing models against each other and against null models generated by label shuffling, based on the 95% CIs for the difference in their performance metrics, computed using the bootstrap method. This approach enabled us to assess the statistical significance of performance differences and to confirm that the models captured meaningful patterns rather than spurious associations or chance‐level variability. The CIs obtained in this way were used to compare our best fluency classifier with (i) the best episodic memory classifier, (ii) the best executive classifier, (iii) the best structural brain classifier, and (iv) the best functional brain classifier. P values were estimated as the fraction of bootstrap differences less than 0, after sign alignment. Differences were deemed significant at an alpha level of 0.05.

2.7. Top fluency features and neural correlates

Neural signatures of the top five discriminatory fluency features were examined through brain–behavior correlations, collapsing AD and HC groups (Figure 1C4), in line with prior approaches. ¹² First, associations with gray matter volume were examined via whole‐brain multiple regressions on SPM12, controlling for TIV, age, education, sex, handedness, acquisition center, and diagnostic group (HC or AD). ⁶² Statistical significance was assessed using threshold‐free cluster enhancement (TFCE) with 5000 permutations, ¹² with an alpha level of p < 0.05, family‐wise error (FWE) corrected. ⁶³ Second, we examined partial Spearman correlations with the internal connectivity of resting‐state fMRI networks yielding significant between‐group differences. Eight networks were considered: (1) default mode, (2) language, (3) executive control, (4) salience/ventral attention, (5) visual, (6) somatomotor, (7) limbic, and (8) memory (Supporting Information 3). Analyses were covaried for sex, age, handedness, years of education, and acquisition center. Statistical significance was set at p < 0.05, FWE‐corrected. ¹²

3. RESULTS

3.1. Group identification across classifiers

As shown in Figure 2, maximal classification was obtained via XGBoost for fluency features (AUC = 0.84 [0.73, 0.90], accuracy = 0.77 [0.67, 0.83], XGBoost for episodic scores (AUC = 0.90 [0.80, 0.96], accuracy = 0.87 [0.79, 0.93]), logistic regression for executive scores (AUC = 0.79 [0.69, 0.87], accuracy = 0.75 [0.65, 0.83]), XGBoost for structural (ROI volume) features (AUC = 0.90 [0.81, 0.95], accuracy = 0.83 [0.74, 0.89]), and logistic regression for functional (connectivity) features (AUC = 0.65 [0.54, 0.74], accuracy = 0.59 [0.51, 0.68]). Full results are provided in Supporting Information 5.

FIGURE 2.

Machine learning results and brain‐behavior correlations. (A) Violin plots show the AUC distribution for each classifier, based on bootstrapped cross‐validation outputs. (B) Feature importance of the best fluency classifier. (C) Clusters showing significant brain–behavior correlations with the top five fluency features. (C1) A negative correlation was observed between word frequency of the semantic task and gray matter volume in right orbitofrontal regions. (C2) A positive correlation was observed between word phonemic length of the semantic task and right orbitofrontal regions. AUC, area under receiver operating characteristic curve.

Fluency‐based classification outcomes did not differ significantly from those obtained via episodic, executive, or structural brain features, as shown by comparing their 95% CIs for AUC values (fluency vs episodic = −0.06 [−0.19, 0.05]; fluency vs executive = −0.05 [−0.07, 0.18]; fluency vs structural = −0.06 [−0.18, 0.06]; all p > 0.14) and for accuracy values (fluency vs episodic = −0.10 [−0.21, 0.00]; fluency vs executive = 0.02 [−0.09, 0.13]; fluency vs structural = −0.06 [−0.19, 0.05]; all p > 0.16). Also, fluency features performed significantly better than the best functional connectivity features (AUC = 0.19 [0.05, 0.33], p = 0.002; accuracy = 0.17 [0.04, 0.28], p = 0.003).

All feature sets yielded chance‐level results upon randomly shuffling participants’ labels, and the five selected classifiers performed significantly better than their respective shuffled versions. For details, see Supporting Information 5.

3.2. Feature‐level analyses

The top five features for the best fluency classifier were word phonemic length, phonological neighborhood, frequency, and granularity in the semantic task and word concreteness in the phonemic task (Figure 2B). Complementary statistical analyses were performed to understand the direction of feature‐level between‐group differences. In the semantic task, relative to HCs, AD patients produced fewer phonated segments, fewer but longer pauses, and a lower speech rate and used words that were significantly more frequent, shorter, more neighbor‐dense, and more conceptually vague (all p < 0.04). No significant differences were observed for the remaining fluency features (all p > 0.063). In the phonemic task, AD patients likewise produced fewer phonated segments, longer pauses, and a lower speech rate and used words that were significantly shorter and more concrete (all p < 0.042), with no other significant differences observed (all p > 0.169). For details, see Supporting Information 6.

3.3. Brain‐behavior correlations

Two of the top five features of the best fluency classifier significantly correlated with participants’ gray matter volume. Word frequency in the semantic task was negatively correlated with the volume of the right orbitofrontal cortex and right inferior frontal gyrus (Figure 2C1, Table 2). Phonemic length in the semantic task was positively correlated with the volume of the right orbitofrontal cortex (Figure 2C2, Table 2). We did not find a significant association between word phonological neighborhood or granularity of the semantic task or word concreteness of the phonemic task and gray matter volume.

TABLE 2.

Brain clusters with significant correlations with the top five fluency features.

Fluency feature	Size (voxels)	Peak region	x	y	z	TFCE	Peak p FWE
Negative: Word logarithmic frequency (semantic task)	1947	Right gyrus rectus	9	33	−26	1413	0.006
		Right superior frontal gyrus, orbital part	9	44	−24	1400	0.006
		Right middle frontal gyrus, orbital part	39	53	0	1382	0.007
Positive: Word phonemic length (semantic task)	351	Right middle frontal gyrus, orbital part	33	51	−14	1064	0.026
			41	45	−14	945	0.041
			41	54	−2	902	0.048

Abbreviation: TFCE, threshold‐free cluster enhancement.

None of the top five fluency features showed significant partial correlations with intrinsic connectivity of the studied networks after controlling for sex, age, handedness, years of education, and acquisition center and after applying FWE correction (all p > 0.09). For details, see Supporting Information 7.

4. DISCUSSION

We benchmarked digital speech biomarkers of AD against standard cognitive and imaging measures. Unlike ST metrics, WP features from fluency tasks robustly classified between patients and HCs. AUC and accuracy metrics did not differ significantly from those obtained via episodic, executive, or neuroanatomical features, and they outperformed functional connectivity features. The two top performing fluency features (length and frequency) were associated with right frontal/orbitofrontal volume. Below we discuss these findings.

The fluency classifier successfully detected AD patients and HCs, yielding an AUC of 0.84. This result closely mirrors prior WP research on AD, ¹² further resembling or surpassing the discriminatory power of multiple digital speech analyses in this population. ⁹ , ¹⁴ , ²³ , ⁶⁴ Importantly, reports of higher classification performance in the speech biomarker literature ⁸ , ⁶⁵ are often based on uninterpretable features from large language models or complex embeddings. Conversely, our approach can be linked to semantic memory difficulties (a hallmark of AD), indicating that patients favor the most retrievable items during vocabulary search. Thus, our fluency framework meets the imperative of balancing sensitivity and interpretability. ⁶⁵

Crucially, outcomes of the fluency classifier were non‐significantly different from those of the best episodic (AUC = 0.90), executive (AUC = 0.79), and structural brain (AUC = 0.90) classifiers, which fell within reported ranges. ³⁰ , ³¹ , ³² , ³³ Of note, the episodic and executive classifiers were trained on several AD‐relevant tasks (targeting immediate and delayed recall, on the one hand, and short‐term memory, working memory, attention, and mental set‐shifting, on the other), while structural classifiers relied on meta‐analytically derived volumetric features. Thus, the three feature sets created stringent benchmarks for comparison, underscoring the robustness of the fluency metrics. Also, the fluency classifier significantly outperformed the functional classifier (AUC = 0.65), which captured resting‐state networks affected in AD. ³ Briefly, then, digital fluency markers meet the non‐inferiority validation criterion relative to well‐established cognitive and imaging measures.

Fluency‐based classification was driven by phonemic length and neighborhood, frequency, granularity, and concreteness. This aligns with works showing that AD patients favor words that are short, neighbor‐dense, frequent, conceptually unspecific, and concrete – namely, items that are more easily retrievable within semantic memory. ¹² , ¹⁶ , ⁶⁶ More particularly, the presence of phonological neighborhood, frequency, and granularity among the top discriminatory features, especially when derived from a semantic fluency task, echoes previous findings, ¹² reinforcing the notion that word access in AD is distinctly characterized by a preference for phonologically crowded, commonly occurring, and conceptually vague words. Also, the preference for shorter words in AD patients has been widely reported, ⁶⁷ and word length alone can discriminate between AD patients and HCs. ¹² More generally, these results underscore WP features as core linguistic signatures of AD.

Two of those features, indeed, correlated with neural patterns. Higher word frequency and lower phonemic length were associated with reduced gray matter volume in the right orbitofrontal cortex, while frequency further correlated with right inferior frontal volume. Both features were previously linked to left‐sided prefrontal activation in other tasks, ⁶⁸ with frequency in word production also yielding associations with the volume of left ¹² , ⁶⁹ and right ⁶⁹ prefrontal regions in AD and right prefrontal regions in behavioral variant frontotemporal dementia. ¹² Although the effects are typically weaker than those observed in the left hemisphere, right prefrontal regions support lexical access and controlled retrieval processes ⁷⁰ and are commonly affected in mild cognitive impairment and AD. ⁷¹ Adding to these antecedents, our findings suggest that atypical word processing in AD may partly reflect right prefrontal disturbances, inviting more studies on these areas’ specific contributions to lexico‐semantic choices in the disorder.

Our results carry clinical implications. Though central to diagnosis, standard cognitive tests typically involve lengthy sessions, highly trained examiners, and expensive licenses, limiting their global deployment, especially in overstretched primary care settings. ⁷² Likewise, MRI evaluations, as well as biofluid tests, prove costly, stressful, and inaccessible in many low‐ and middle‐income countries. ⁷³ Conversely, much like other digital speech biomarkers, the present fluency framework is automated, time‐efficient, affordable, patient‐friendly, and remotely deployable, attesting to its scalability. ⁷ , ¹⁴ Indeed, our ST and WP features are already integrated into apps currently used in clinical centers worldwide. ¹⁹ , ²⁰ Thus, just as novel blood tests are maximizing the scalability of pathological markers, ⁷⁴ so too can fluency measures boost the global deployment of cognitive markers – all while matching the discriminability of standard tools. This is particularly relevant in Latin America and other underresourced regions, where structural limitations and high costs substantially restrict access to traditional diagnostic methods. ⁵ , ⁶ Overall, by focusing on Latino participants, this study fosters equity and diversity in speech‐based AD research, beyond its predominant focus on English‐speaking cohorts. ⁷

Despite its strengths, this study has several limitations. Although our sample size (n = 66) was similar to the median of previous digital speech marker studies (median n = 70, derived from a systematic review), ⁶⁴ replications with larger cohorts for robust generalizability tests are needed. This limitation also restricted the exploration of broader feature sets, given feature‐to‐sample ratio constraints. ³⁷ Future research with more participants may evaluate classifiers trained on more comprehensive speech, cognitive, and imaging feature sets. Such replications would also benefit from additional measures absent in our protocol. These include validated tools drawing on social determinants of health ⁷⁵ and biofluid markers known to modulate clinical outcomes in AD. ⁴ Moreover, patients were in moderate disease stages, calling for replications in mild‐stage cohorts to better establish the utility of WP and ST markers for early detection. Also, although the similarities with Ferrante et al.’s (2024) study ¹² are noteworthy, roughly 36% of our sample overlapped with the one reported therein. Future studies should replicate our approach in a fully distinct sample to better ascertain the generalizability of the observed patterns. Finally, the episodic memory classifier was based on tests that differed among participants. While this was tackled via well‐established normalization methods, future works should replicate our approach with consistent (and, ideally, additional) episodic memory measures.

In sum, our findings reinforce digital fluency metrics as robust AD markers, on par with standard cognitive and imaging measures. Importantly, relative to such benchmarks, this framework proves more affordable, time‐efficient, and scalable, thereby addressing dire needs of low‐resource, underserved settings. Future extensions of this approach, especially in broader primary care cohorts, could yield additional validation for its widespread real‐world deployment.

AUTHOR CONTRIBUTIONS

Iván Caro: Methodology, formal analysis, writing – original draft. Gonzalo Pérez: Methodology, software, formal analysis, writing – original draft. Joaquín Valdés Bize: Formal analysis, writing – original draft. Joaquín Ponferrada: Software, writing – review and editing. Franco Ferrante: Software, writing – review and editing. Alejandro Sosa Welford: Software, writing – review and editing. Lara Gauder: Data curation, writing—review & editing. Loreto Olavarría: Validation, writing – review and editing. Fernando Henríquez: Validation, writing – review and editing. Teresita Ramos: Validation, writing – review and editing. Cristina Besnier: Validation, writing – review and editing. Luciana Ferrer: Methodology, writing – review and editing. Maria Luisa Gorno‐Tempini: Writing – review and editing. Andrea Slachevsky: Resources, data curation, writing – review and editing. Agustín Ibañez: Resources, funding acquisition, writing – review and editing. Adolfo M. García: Conceptualization, writing – review and editing, supervision, project administration, funding acquisition.

CONFLICT OF INTEREST STATEMENT

Adolfo M. García is CSO of TELL. All other authors report no conflicts of interest. Author disclosures are available in the Supporting Information.

CONSENT STATEMENT

All participants of the study provided written informed consent in accordance with the Declaration of Helsinki. The study was approved by the Institutional Ethics Committee CEC SSM Oriente under code 29092020.

Supporting information

Supporting information

Supporting information

DATA AVAILABILITY STATEMENT

All anonymized data and scripts supporting the findings of this study are freely available in the following repository: https://osf.io/ad8jb/overview?view_only=e69e33184e334282ade387b1c4da32a9.