Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 May 1.
Published in final edited form as: Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2023 Jun 5;31(3):575–586. doi: 10.1080/13825585.2023.2221020

Distinguishable features of spontaneous speech in Alzheimer’s clinical syndrome and healthy controls

Erin Burke 1, John Gunstad 1, Olesia Pavlenko 2, Phillip Hamrick 1
PMCID: PMC10696129  NIHMSID: NIHMS1906258  PMID: 37272884

Abstract

There is growing evidence that subtle changes in spontaneous speech may reflect early pathological changes in cognitive function. Recent work (Ostrand and Gunstad, 2021) found that lexical-semantic features of spontaneous speech predict cognitive dysfunction in individuals with mild cognitive impairment (MCI). The current study assessed whether Ostrand and Gunstad’s (OG) lexical-semantic features extend to predicting cognitive status in a sample of individuals with Alzheimer’s clinical syndrome (ACS) and healthy controls. Four additional (New) speech indices shown to be important in language processing research were also explored in this sample to extend prior work. Speech transcripts of the Cookie Theft Task from 81 individuals with ACS (Mage = 72.7 years, SD = 8.80, 70.4% female) and 61 healthy controls (HC) (Mage = 63.9 years, SD = 8.52, 62.3% female) from Dementia Bank were analyzed. Random forest and logistic machine learning techniques examined whether subject-level lexical-semantic features could be used to accurately discriminate those with ACS from HC. Results showed that logistic models with the New lexical-semantic features obtained good classification accuracy (78.4%), but the OG features had wider success across machine learning model types. In terms of sensitivity and specificity, the random forest model trained on the OG features was the most balanced. Findings from the current study suggest that features of spontaneous speech used to predict MCI may also distinguish between individuals with ACS and healthy controls. Future work should evaluate these lexical-semantic features in pre-clinical persons to further explore their potential to assist with early detection through speech analysis.

Keywords: spontaneous speech, Alzheimer’s Disease (AD), Alzheimer’s clinical syndrome, machine learning

Introduction

Machine learning techniques have become increasingly utilized to examine spontaneous speech in persons with Mild Cognitive Impairment (MCI) and Alzheimer’s disease (AD) (Fraser et al., 2016; Hernández-Domínguez et al., 2018; Lindsay et al., 2021; Rentoumi et al., 2014), as disruptions across multiple linguistic levels (i.e., phonetic, phonological, lexical-semantics, morphosyntactic, and pragmatic) are well-established in these conditions (Boschi et al., 2017; Bayles et al., 1992; Fleming & Harris, 2008; Pistono et al, 2019; Pistono et al., 2016; Ostrand & Gunstad, 2021; Roark et al., 2011; Szatloczki et al., 2015; Taler & Phillips, 2008). Specifically, word finding difficulty (Nelson & O’Connor, 2008; Yeung et al., 2021) and decreased lexical diversity (Ostrand and Gunstad, 2021) are characteristic features of MCI, and additional declines in semantic, syntactic, and lexical functions are found as these individuals progress to AD (Ahmed et al., 2013). AD is further characterized by decreased semantic content and syntactical complexity, empty speech (i.e., a lack of descriptive specificity and the use of “thing,” or “stuff”) (Forbes-McKay et al., 2013), reduced speech rate (Hoffmann et al., 2010), greater use of pronouns and indefinite terms (Ahmed et al., 2013; Lai et al., 2014) and repetition (Sajjadi et al., 2012).

Recent work raises the possibility that these linguistic indices derived from spontaneous speech might be more sensitive to pathological cognitive decline than traditional language tests of confrontation naming and semantic fluency (Bird et al., 2000; Forbes-McKay et al., 2013; Henry et al., 2004; Kavé & Levy, 2003; Loewenstein et al., 2018; Mueller et al., 2016; Ostrand & Gunstad, 2021; Sajjadi et al., 2012; Slegers et al., 2018; Szatloczki et al., 2015; Taler & Phillips, 2008; Vonk et al., 2022). However, despite being identified as an important next step for the field (e.g., Boschi et al., 2017; Bucks et al., 2000; Filiou et al., 2020; Mueller et al., 2016), very few studies have examined whether lexical-semantic features found to be sensitive to MCI can be cross validated to AD samples. This lack of replication is a limitation to understanding how speech indices change over time in the context of cognitive decline.

The current study had two primary goals. First, lexical-semantic features found to distinguish persons with MCI from healthy controls in past work (Ostrand & Gunstad, 2021) were examined in a new sample of persons with Alzheimer’s clinical syndrome to determine the stability and generalizability of these features. In addition, several other lexical-semantic features not previously investigated in neurodegenerative populations were utilized to help provide a more comprehensive assessment of spontaneous speech in this population. Although conceptually similar to some of the OG features, these novel features have been shown to be particularly powerful in other studies of language in AD (e.g., Hernandez-Dominguez et al., 2018) and in language processing more broadly (e.g., Hamrick & Pandža, 2020). It was hypothesized that the lexical-semantic features shown to be sensitive to MCI would also be sensitive to Alzheimer’s clinical syndrome and that the inclusion of additional speech indices of lexical-semantic functioning would increase predictive validity. If confirmed, these findings would provide further evidence for the idea that spontaneous speech assessment can be useful in differentiating between normative speech changes and neurodegenerative conditions.

Methods

Corpus

We used publicly available data from the Pitt Corpus (Becker et al., 1994) of the DementiaBank database. This corpus contains manual transcriptions of participants who completed a standard version of the Cookie Theft picture description task from the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983). This archive contains data from healthy controls, subjects with a variety of dementias, including “probable” and “possible” AD, MCI, vascular dementia, and Parkinson’s disease. Participant history of cognitive and functional decline obtained via extensive interviews, medical workup, and neuropsychological performance completed at the time of assessment were used to inform diagnostic groups (see Becker et al., 1994 for more details regarding diagnostic procedures). To avoid confounds from other diagnoses on language production, we only examined data from participants who were either labeled by the original researchers as “probable AD” with no other diagnoses (n = 81) and healthy controls (n = 61). Since there is a lack of reported biomarkers associated with the “probable AD” classification in the Pitt Corpus, we use the term “Alzheimer’s clinical syndrome” (ACS) here instead of “AD” or “probable AD.” Moreover, because the aim of this research is to ultimately be able to predict AD as early as possible, we opted to focus on participants’ first visit transcripts rather than looking at all their transcripts across several visits.

Within this sample, healthy controls (63.90 ± 8.53 years) were younger than persons in the ACS group (72.70 ± 8.80); t(140) = 5.98, p <.001), but did not differ in gender [62.3% women vs 70.4% women t(140) = 1.009, p = .32]. Given this pattern, the potential contribution of age to study findings was directly examined as part of analysis.

Lexical-semantic features

We were able to generate 13 of the 16 lexical-semantic features computed by Ostrand and Gunstad (2021). Three of their lexical-semantic features (Empty Words, Speech Rate, Filler Rate) were not able to be computed on the Pitt Corpus data due to missing information. The 13 variables included were Total Number of Words, Filler Words, Word Frequency, Type-Token Ratio, Honore’s Statistic, Brunet’s Index, Definite Articles, Content Words, Indefinite Articles, Pronouns, Nouns, Verbs, and Determiners. Their definitions and the details of their computation can be found in Table 2 and will be hereafter be referred to as the OG features.

Table 2.

OG Speech Indices Operational Definition
Total Words total number of words spoken by the subject
Fillers number filler words (e.g., um, uh, hmm) spoken by the subject, scaled by the total word count
Definites total number of definite articles (the), scaled by the total word count
Indefinites total number of indefinite articles (a, an), scaled by the total word count
Pronouns number of pronouns (calculated by the Penn Treebank POS tags), scaled by the total word count
Nouns number of nouns (calculated by the Penn Treebank POS tags), scaled by the total word count
Verbs number of verbs (calculated by the Penn Treebank POS tags), scaled by the total word count
Determiners number of determiners (as calculated by the Penn Treebank POS tags), scaled by the total word count
Content Words number of content words (defined by the words NOT in NLTK's list of stop words), scaled by word count
Frequency mean of the log of the frequency of all the words spoken by the subject
Token Ratio number of different word types accounting for total number of words (a measure of vocabulary size and lexical richness)
Honore’s Statistic a measure of lexical richness/diversity (number of words produced exactly once). It is calculated as: (100 * log(tokens)) / (1 − V1/types), where V1 is the number of words spoken exactly once
Brunet’s index a length-insensitive measure of lexical diversity/richness. It is calculated as: tokens ^ types ^ (−0.165)
New Speech Indices Operational Definition
Hapax Legomena another way of measuring lexical richness of vocabulary (simple count of the number of words that are produced exactly once)
HD Type-Token Ratio ratio of each participanťs Hapax Legomena score to the total number of unique base words
Root Type Token Ratio number of unique words divided by the square root of the total number of words (like TTR, but addresses text length variation)
Semantic Distinctiveness a measure of semantic diversity of a word (number of different semantic contexts in which a word appears). e−λ*cos(context,wordi)
Open in a new tab

In addition to attempting to replicate the OG features in persons with ACS, we also included four variables (New features) which have been shown to be useful in both AD research as well as psycholinguistic research. These four lexical-semantic features were: Hapax Legomena, HD Type-Token Ratio (computed following Hernandez-Dominguez et al., 2018), Root Type-Token Ratio, and Semantic Distinctiveness. Hapax Legomena refers to the number of words (lemmas) used only once within a participant’s speech transcript, with a larger value reflecting greater lexical diversity. HD Type-Token Ratio was computed following Hernandez-Dominguez et al. (2018) and was the ratio of each participant’s Hapax Legomena score to their total number of unique lemmas. Root Type-Token Ratio was computed as the number of unique word types divided by the square root of the total number of tokens. Finally, semantic distinctiveness was introduced as an alternative word frequency metric. In short, traditional word frequency metrics are built around the idea that a word’s strength in memory is a function of the number of times it is repeated; however, increasing evidence suggests that the number of contexts in which a word occurs is a better predictor (Adelman et al., 2006) of a word’s strength in memory, and this value is made even more predictive if it is weighted by the semantic distinctiveness of those contexts (Jones et al., 2012), and this holds in monolinguals (Jones et al., 2012), bilinguals (Hamrick & Pandža, 2020), and in aging (Johns et al., 2016). These lexical-semantic features will hereafter be referred to as the “New” features.

Classification modeling with machine learning

Machine learning is a subfield of artificial intelligence that employs automatic model construction to solve, among other things, classification problems. In this study, we employed two popular model types: logistic regression and random forest classifiers.1 The code and data for these models is available at https://dementia.talkbank.org/. We used two types of classifiers because each has complementary strengths and weaknesses. Moreover, if we found similar results across the classifier types, it could be taken as evidence of more robustness in the roles of our lexical-semantic features in ACS classification. The model training and testing procedure was as follows. The entire dataset consisted of subject-level values for the 17 lexical-semantic features along with each participant’s diagnosis data (ACS vs HC). The entire dataset from all 142 participants was randomly split into two sets: a training/evaluation dataset (75% of all data) and a testing dataset (the remaining 25% of the data). The training/evaluation dataset was then randomly divided into 25 bootstrap resampled datasets. Within each of these resample training/evaluation datasets, we bootstrap resampled another 75/25% split, with 75% of each resample being used for model training purposes and the other 25% being used to evaluate the overall model fit. This procedure was repeated across each of the 25 training/evaluation datasets. In the final step of modelling, we then tested the best overall model from these fits against the testing dataset, which consisted exclusively of data that had been held aside at the outset and that none of the models had ever been exposed to prior. Note, that this modeling procedure establishes thresholds for classification of participants as ACS and HC based on the models that best fit the training data, and no further model tuning or boosting was used to alter these thresholds, so as to avoid potential issues with overfit. In all cases, resampling was stratified to preserve similar proportions of ACS and HC participants in all datasets. These final model fits to the test data set were evaluated based on the accuracy of classification (ACS vs HC) as well as ROC-AUC. All analyses were conducted using the tidymodels package (Kuhn & Wickham, 2021) within the R statistical programming language (R Core Team, 2021), and the code for reproducing this modeling procedure is publicly available (https://osf.io/qc3fw/?view_only=8406d0372b68472486f915ce25e455cf).

Results

The models’ abilities to accurately classify test (i.e., generalization) participants as either ACS or HC were evaluated on the basis of their accuracy, area under the curve of receiver operating characteristics (AUC, see Figures 1 and 2), their sensitivity (i.e., the % of time the model classifies ACS correctly), and their specificity (i.e., the % of time HC was classified correctly). The results are shown in Table 3, which shows better performance for random forest over logistic models for the OG lexical-semantic features, as well as better performance for logistic models over random forest models in the New lexical-semantic features.

Figure 1.

Figure 1.

Open in a new tab

ROC-AUC for the logistic and random forest classifier models for the OG features.

Figure 2.

Figure 2.

Open in a new tab

ROC-AUC for the logistic and random forest classifier models for the New features.

Table 3.

Performance of classifier models at distinguishing ACS patients and HCs.

Model Type Features Accuracy AUC Sensitivity Specificity
Logistic OG 0.649 0.708 0.571 0.750
Random Forests OG 0.757 0.804 0.761 0.750
Logistic New 0.784 0.830 0.857 0.687
Random Forests New 0.486 0.533 0.523 0.437
Open in a new tab

Abbreviations: ACS, Alzheimer’s clinical syndrome; AUC = area under the curve of receiver operating characteristics; HCs, healthy elderly controls; OG = 13 semantic features used in Ostrand and Gunstad (2021) re-computed over the current dataset.

Accuracy and AUC values were highest for the logistic model based on the New lexical-semantic features, suggesting that those lexical-semantic features may have particularly promising value for future research (see figure 1 for a visual of these findings). At the same time, the model with the most balanced sensitivity and specificity was the random forest model trained on the OG lexical-semantic features.

Because age differed between the ACS and HC groups, we also ran the entire modeling procedure described above while including age as a covariate in the models (see Table 4). This approach resulted in a similar overall pattern of findings (e.g., random forest models were superior to logistic models for the OG features, but the converse was true for the New features), with generally better fit to the data - which is unsurprising given that age is correlated with ACS diagnosis.

Table 4.

Performance of classifier models at distinguishing ACS patients and HCs, with age included as a covariate in the model.

Model Type Features Accuracy AUC Sensitivity Specificity
Logistic OG 0.703 0.810 0.667 0.750
Random Forests OG 0.865 0.836 0.857 0.875
Logistic New 0.703 0.839 0.761 0.625
Random Forests New 0.595 0.685 0.571 0.625
Open in a new tab

Abbreviations: ACS, Alzheimer’s clinical syndrome; AUC = area under the curve of receiver operating characteristics; HCs, healthy elderly controls; OG = 13 semantic features used in Ostrand and Gunstad (2021) re-computed over the current dataset.

Discussion

Findings from the current study both replicate and extend past work. Features of spontaneous speech shown to distinguish persons with MCI from healthy older adults in past work (Ostrand & Gunstad, 2021) successfully identified persons with ACS in the current sample. The inclusion of additional indices from more recent work in language processing research increased predictive validity over previously used lexical-semantic features. Several aspects of these findings warrant brief discussion.

Finding that OG speech indices successfully predicted group status in the current study (i.e., ACS vs healthy control) is a modest but valuable contribution to the literature. As described above, though a growing number of studies have identified lexical-semantic features that can distinguish persons with and without neurological conditions (e.g., Boschi et al., 2017; Fraser et al., 2016; Ostrand & Gunstad, 2021; Roark et al., 2011), very few studies have examined whether the same set of speech features found to be sensitive to impairment in one sample generalize to another (Bucks et al., 2000). This makes interpretation of inconsistent findings difficult, as differences across studies may then be due to either conceptual issues (e.g., incomplete or incorrect understanding of language in AD) or any number of methodological choices (e.g., specific lexical-semantic features being examined, sample composition, statistical techniques, etc.). Our results showing lexical-semantic features that predict MCI also predict ACS support the notion that these indices have the capacity to detect lexical-semantic features characteristic of cognitive decline (Ahmed et al., 2013; Forbes-McKay et al., 2013; Ostrand & Gunstad, 2021) and that MCI and ACS share similar speech characteristics (Boschi et al., 2017; Taler & Philips, 2008). Future work should continue to cross-validate lexical-semantic indices across samples to further support their use in identifying changes in speech associated with neurodegenerative conditions.

In addition to generalizing findings from past work, the current analyses also revealed that four other markers of lexical and semantic performance were sensitive to ACS and distinguished these persons from healthy controls. These indices (i.e., Hapax Legomena, HD Type-Token Ratio, Root Type-Token Ratio, and Semantic Distinctiveness) are similar to the OG indices in that they also measure lexical-semantic diversity in spoken language. The fact that these lexical-semantic features were also disrupted in the context of neurodegeneration represents an important confirmation that both MCI and ACS affect lexical-semantic processing broadly, not just for some lexical-semantic features. Such findings are consistent with past work showing numerous aspects of speech and language function may rely on the declarative memory system (Hamrick, Lum, & Ullman, 2018) that are disrupted in persons with MCI/AD and encourage continued examination of other elements including acoustic features such as voice quality and pauses (Gosztolya et al., 2019; Pistono et al., 2016; Themistocleous et al., 2020).

The fact that the lexical-semantic features had different levels of classification performance is not surprising, as logistic and random forest models often differ in their success rates, largely as a function of the datasets used, the nature of the predictor variables and their multicollinearity, and a host of other factors (Couronné, Probst, & Boulesteix, 2018). Future research should examine the differential efficacy of random forest and logistic models (as well as other common machine learning techniques, such as support vector machines) in using lexical-semantic features to predict AD classification, especially since modern machine learning techniques involve a range of tuning parameters. At their best, principle tuning of these parameters may result in optimal classification performance, but at their worst, parameter tuning could be fishing for statistical significance. More research on this issue is needed.

Using existing, archived data limited the characterization of diagnostic groups. Cognition of participants was not comprehensively assessed and biomarker confirmation of diagnostic groups was not available from that original project, but would serve as key components to further understanding in future work (Graff-Radford et al., 2021). For example, individuals with AD recruit different brain regions on tasks of semantic and working memory compared to healthy controls (Hirni et al., 2013; Teipel et al., 2015), raising the possibility that neuroimaging correlates of spontaneous speech may be detectable at various disease stages. Using neuroimaging to investigate the association between atrophy or amyloid deposits to spontaneous speech performance may shed key insight into the biological representations of changes in spontaneous speech. Some evidence for these relationships already exists. Past work reveals atrophy in cortical regions critical for language processes (i.e., naming, semantic fluency and word retrieval) such as the hippocampus and Broadmann’s areas 37 and 40, as well as reduced activation in prefrontal and parietal regions (Harasty et al., 1999; Hirni et al., 2013; McGeown et al., 2009; Venneri et al., 2008). Reduced activation and grey matter volume existing in these language areas may reflect impaired spontaneous speech that occurs in relation to deficits in semantic memory seen in AD (Zannino et al., 2015).

Future work should also prospectively investigate the sensitivity and specificity of these lexical-semantic features to AD in larger, age- and demographically-balanced samples. Such work would provide key insight into speech changes associated with normal and pathological cognitive aging, as well as provide a better understanding of possible influence of age, gender, and race/ethnicity. For example, the wide age range of the current sample (i.e., 47 to 88) limits the opportunity to generate insight into speech within narrow age ranges (e.g., 40–50 years of age vs 50–60 years of age). Further, the lack of racial/ethnic diversity in the current sample raises concerns for understanding known cultural influences on language (Gutchess et al., 2010). Similarly, further work into possible sex differences is also needed, as women exhibit greater atrophy in brain regions important for language (Harastay et al., 1999) and the potential contribution of these changes to spontaneous speech is poorly understood. Additionally, though picture description tasks such as Cookie Theft are successful in revealing linguistic impairments (Cummings, 2019), it is possible that other tasks may generate speech samples even more sensitive to AD. Picture description tasks are highly structured and rely on a lower cognitive load relative to expository speech tasks (i.e., responding to open-ended questions in an interview or telling a story). Examination of the ability of the speech indices utilized in the current study to predict ACS status from other speech samples appears warranted.

In brief summary, lexical-semantic features sensitive to MCI in past work were also sensitive to ACS in the current sample and novel markers of lexical and semantic distinctiveness showed incremental validity in the identification of persons with ACS. The current findings encourage further examination of the possible utility of automated lexical-semantic analyses to aid in early detection of MCI and AD.

Table 1.

Biographical data M(SD)

Age at first visit MMSE Score Sex (female/male) Education
ACS 72.70 (8.80) 19.42 (4.47) 57/24 11.80 (2.91)
Healthy Controls 63.90 (8.52) 29.08 (1.05) 38/23 13.89 (2.30)
Open in a new tab

Abbreviations: ACS, Alzheimer’s clinical syndrome; MMSE, Mini Mental State Examination

Acknowledgments

Data for the current study was extracted from DementiaBank, which is funded by NIH grants NIA AG03705 and AG05133. Dr. Gunstad funded in part by NIH (NIA R01AG065432).

Footnotes

1

These two methods were used as complementary. Some studies have suggested superior performance for random forest over logistic models (Maroco, Silva, Rodrigues, Guerreiro, Santana, & Mendonça, 2011), while others have shown better performance in logistic models (Kirasich, Smith, & Sadler, 2018). Because random forest models are difficult to interpret in terms of the structure of their resulting models and involve fine-tuning of parameters (which we left standard in our analyses, including the default number of trees), we opted to also use logistic models, given that they are more interpretable and may provide, in some cases, superior fit.

REFERENCES

  1. Adelman JS, Brown GD, & Quesada JF (2006). Contextual diversity, not word frequency, determines word-naming and lexical decision times. Psychological science, 17(9), 814–823. 10.1111/j.1467-9280.2006.01787.x [DOI] [PubMed] [Google Scholar]
  2. Ahmed S, Haigh AM, de Jager CA, & Garrard P (2013). Connected speech as a marker of disease progression in autopsy-proven Alzheimer’s disease. Brain: a journal of neurology, 136(Pt 12), 3727–3737. 10.1093/brain/awt269 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bayles KA, Tomoeda CK, & Trosset MW (1992). Relation of linguistic communication abilities of Alzheimer’s patients to stage of disease. Brain and language, 42(4), 454–472. 10.1016/0093-934x(92)90079-t [DOI] [PubMed] [Google Scholar]
  4. Becker JT, Boller F, Lopez OL, Saxton J, & McGonigle KL (1994). The natural history of Alzheimer’s disease. Description of study cohort and accuracy of diagnosis. Archives of neurology, 51(6), 585–594. 10.1001/archneur.1994.00540180063015 [DOI] [PubMed] [Google Scholar]
  5. Bird H, Ralph MAL, Patterson K, & Hodges JR (2000). The rise and fall of frequency and imageability: Noun and verb production in semantic dementia. Brain and language, 73(1), 17–49. 10.1006/brln.2000.2293 [DOI] [PubMed] [Google Scholar]
  6. Boschi V, Catricalà E, Consonni M, Chesi C, Moro A, & Cappa SF (2017). Connected Speech in Neurodegenerative Language Disorders: A Review. Frontiers in psychology, 8, 269. 10.3389/fpsyg.2017.00269 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bucks RS, Singh S, Cuerden JM, & Wilcock GK (2000). Analysis of spontaneous, conversational speech in dementia of Alzheimer type: Evaluation of an objective technique for analysing lexical performance. Aphasiology, 14(1), 71–91. 10.1080/026870300401603 [DOI] [Google Scholar]
  8. Clarke N, Foltz P, & Garrard P (2020). How to do things with (thousands of) words: Computational approaches to discourse analysis in Alzheimer’s disease. Cortex, 129, 446–463. 10.1016/j.cortex.2020.05.001 [DOI] [PubMed] [Google Scholar]
  9. Couronné R, Probst P, & Boulesteix AL (2018). Random forest versus logistic regression: a large-scale benchmark experiment. BMC bioinformatics, 19(1), 270. 10.1186/s12859-018-2264-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cummings L (2019). Describing the cookie theft picture: Sources of breakdown in alzheimer’s dementia. Pragmatics and Society, 10(2), 153–176. 10.1075/ps.17011.cum [DOI] [Google Scholar]
  11. Goodglass H and Kaplan E (1983). Boston diagnostic aphasia examination booklet. Lea & Febiger; Philadelphia, PA. [Google Scholar]
  12. Gosztolya G, Vincze V, Tóth L, Pákáski M, Kálmán J, & Hoffmann I (2019). Identifying mild cognitive impairment and mild Alzheimer’s disease based on spontaneous speech using ASR and linguistic features. Computer Speech & Language, 53, 181–197. [Google Scholar]
  13. Gutchess AH, Hedden T, Ketay S, Aron A, & Gabrieli JD (2010). Neural differences in the processing of semantic relationships across cultures. Social cognitive and affective neuroscience, 5(2–3), 254–263. 10.1093/scan/nsp059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Graff-Radford J, Yong KXX, Apostolova LG, Bouwman FH, Carrillo M, Dickerson BC, Rabinovici GD, Schott JM, Jones DT, & Murray ME (2021). New insights into atypical Alzheimer’s disease in the era of biomarkers. The Lancet. Neurology, 20(3), 222–234. 10.1016/S1474-4422(20)30440-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Filiou RP, Bier N, Slegers A, Houzé B, Belchior P, & Brambati SM (2020). Connected speech assessment in the early detection of Alzheimer’s disease and mild cognitive impairment: a scoping review. Aphasiology, 34(6), 723–755. 10.1080/02687038.2019.1608502 [DOI] [Google Scholar]
  16. Fleming VB, & Harris JL (2008). Complex discourse production in mild cognitive impairment: detecting subtle changes. Aphasiology, 22(7–8), 729–740. 10.1080/02687030701803762 [DOI] [Google Scholar]
  17. Forbes-McKay K, Shanks MF, & Venneri A (2013). Profiling spontaneous speech decline in Alzheimer’s disease: a longitudinal study. Acta neuropsychiatrica, 25(6), 320–327. 10.1017/neu.2013.16 [DOI] [PubMed] [Google Scholar]
  18. Fraser KC, Meltzer JA, & Rudzicz F (2016). Linguistic Features Identify Alzheimer’s Disease in Narrative Speech. Journal of Alzheimer’s disease : JAD, 49(2), 407–422. 10.3233/JAD-150520 [DOI] [PubMed] [Google Scholar]
  19. Hamrick P, Lum JAG, & Ullman MT (2018). Child first language and adult second language are both tied to general-purpose learning systems. Proceedings of the National Academy of Sciences, 115, 1487–1492. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hamrick P, & Pandža NB (2020). Contributions of semantic and contextual diversity to the word frequency effect in L2 lexical access. Canadian Journal of Experimental Psychology, 74, 25–34. 10.1037/cep0000189 [DOI] [PubMed] [Google Scholar]
  21. Harasty JA, Halliday GM, Kril JJ, & Code C (1999). Specific temporoparietal gyral atrophy reflects the pattern of language dissolution in Alzheimer’s disease. Brain : a journal of neurology, 122 ( Pt 4), 675–686. 10.1093/brain/122.4.675 [DOI] [PubMed] [Google Scholar]
  22. Henry JD, Crawford JR, & Phillips LH (2004). Verbal fluency performance in dementia of the Alzheimer’s type: A meta-analysis. Neuropsychologia, 42(9), 1212–1222. 10.1016/j.neuropsychologia.2004.02.001 [DOI] [PubMed] [Google Scholar]
  23. Hernández-Domínguez L, Ratté S, Sierra-Martínez G, & Roche-Bergua A (2018). Computer-based evaluation of Alzheimer’s disease and mild cognitive impairment patients during a picture description task. Alzheimer’s & dementia (Amsterdam, Netherlands), 10, 260–268. 10.1016/j.dadm.2018.02.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hirni DI, Kivisaari SL, Monsch AU, & Taylor KI (2013). Distinct neuroanatomical bases of episodic and semantic memory performance in Alzheimer’s disease. Neuropsychologia, 51(5), 930–937. 10.1016/j.neuropsychologia.2013.01.013 [DOI] [PubMed] [Google Scholar]
  25. Hoffmann I, Nemeth D, Dye CD, Pákáski M, Irinyi T, & Kálmán J (2010). Temporal parameters of spontaneous speech in Alzheimer’s disease. International journal of speech-language pathology, 12(1), 29–34. 10.3109/17549500903137256 [DOI] [PubMed] [Google Scholar]
  26. Johns BT, Sheppard CL, Jones MN, & Taler V (2016). The Role of Semantic Diversity in Word Recognition across Aging and Bilingualism. Frontiers in psychology, 7, 703. 10.3389/fpsyg.2016.00703 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Jones MN, Johns BT, & Recchia G (2012). The role of semantic diversity in lexical organization. Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale, 66(2), 115–124. 10.1037/a0026727 [DOI] [PubMed] [Google Scholar]
  28. Kavé G, & Goral M (2016). Word retrieval in picture descriptions produced by individuals with Alzheimer’s disease. Journal of clinical and experimental neuropsychology, 38(9), 958–966. 10.1080/13803395.2016.1179266 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kavé G, & Levy Y (2003). Morphology in picture descriptions provided by persons with Alzheimer’s disease. Journal of Speech, Language, and Hearing Research, 46(2), 341–352. 10.1044/1092-4388(2003/027) [DOI] [PubMed] [Google Scholar]
  30. Kirasich K, Smith T, & Sadler B (2018). Random forest vs logistic regression: Binary classification for heterogeneous datasets. SMU Data Science Review, 1(3), article 9. [Google Scholar]
  31. Kuhn M, & Wickham H (2020). Tidymodels: A collection of packages for modeling and machine learning using tidyverse principles. https://www.tidymodels.org/
  32. Lai YH (2014). Discourse features of Chinese-speaking seniors with and without Alzheimer’s disease. Language and Linguistics, 15(3), 411–434. 10.1177/1606822X14520665 [DOI] [Google Scholar]
  33. Lindsay H, Tröger J, & König A (2021). Language Impairment in Alzheimer’s Disease-Robust and Explainable Evidence for AD-Related Deterioration of Spontaneous Speech Through Multilingual Machine Learning. Frontiers in aging neuroscience, 13, 642033. 10.3389/fnagi.2021.642033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Loewenstein DA, Curiel RE, Duara R, & Buschke H (2018). Novel Cognitive Paradigms for the Detection of Memory Impairment in Preclinical Alzheimer’s Disease. Assessment, 25(3), 348–359. 10.1177/1073191117691608 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Maroco J, Silva D, Rodrigues A Guerreiro M, Santana I, & Mendonça A (2011). Data mining methods in the prediction of Dementia: A real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests. BMC Research Notes, 4, 299. 10.1186/1756-0500-4-299 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. McGeown WJ, Shanks MF, Forbes-McKay KE, & Venneri A (2009). Patterns of brain activity during a semantic task differentiate normal aging from early Alzheimer’s disease. Psychiatry research, 173(3), 218–227. 10.1016/j.pscychresns.2008.10.005 [DOI] [PubMed] [Google Scholar]
  37. Mueller KD, Koscik RL, Turkstra LS, Riedeman SK, LaRue A, Clark LR, Hermann B, Sager MA, & Johnson SC (2016). Connected language in late middle-aged adults at risk for Alzheimer’s disease. Journal of Alzheimer’s Disease, 54(4), 1539–1550. 10.3233/JAD-160252 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Nelson AP, & O’Connor MG (2008). Mild cognitive impairment: a neuropsychological perspective. CNS spectrums, 13(1), 56–64. 10.1017/s1092852900016163 [DOI] [PubMed] [Google Scholar]
  39. Ostrand R, & Gunstad J (2021). Using Automatic Assessment of Speech Production to Predict Current and Future Cognitive Function in Older Adults. Journal of geriatric psychiatry and neurology, 34(5), 357–369. 10.1177/0891988720933358 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Pistono A, Jucla M, Barbeau EJ, Saint-Aubert L, Lemesle B, Calvet B, Köpke B, Puel M, & Pariente J (2016). Pauses during autobiographical discourse reflect episodic memory processes in early Alzheimer’s disease. Journal of Alzheimer’s Disease, 50(3), 687–698. 10.3233/JAD-150408 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Pistono A, Pariente J, Bézy C, Lemesle B, Le Men J, & Jucla M (2019). What happens when nothing happens? An investigation of pauses as a compensatory mechanism in early Alzheimer’s disease. Neuropsychologia, 124, 133–143. 10.1016/j.neuropsychologia.2018.12.018 [DOI] [PubMed] [Google Scholar]
  42. R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/ [Google Scholar]
  43. Rentoumi V, Raoufian L, Ahmed S, de Jager CA, & Garrard P (2014). Features and machine learning classification of connected speech samples from patients with autopsy proven Alzheimer’s disease with and without additional vascular pathology. Journal of Alzheimer’s disease : JAD, 42 Suppl 3, S3–S17. 10.3233/JAD-140555 [DOI] [PubMed] [Google Scholar]
  44. Roark B, Mitchell M, Hosom JP, Hollingshead K, & Kaye J (2011). Spoken Language Derived Measures for Detecting Mild Cognitive Impairment. IEEE transactions on audio, speech, and language processing, 19(7), 2081–2090. 10.1109/TASL.2011.2112351 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Sajjadi SA, Patterson K, Tomek M, & Nestor PJ (2012). Abnormalities of connected speech in semantic dementia vs Alzheimer’s disease. Aphasiology, 26(6), 847–866. 10.1080/02687038.2012.654933 [DOI] [Google Scholar]
  46. Slegers A, Filiou R-P, Montembeault M, & Brambati SM (2018). Connected Speech Features from Picture Description in Alzheimer’s Disease: A Systematic Review. Journal of Alzheimer’s Disease, 65(2), 519–542. 10.3233/JAD-170881 [DOI] [PubMed] [Google Scholar]
  47. Szatloczki G, Hoffmann I, Vincze V, Kalman J, & Pakaski M (2015). Speaking in Alzheimer’s Disease, is That an Early Sign? Importance of Changes in Language Abilities in Alzheimer’s Disease. Frontiers in aging neuroscience, 7, 195. 10.3389/fnagi.2015.00195 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Taler V, & Phillips NA (2008). Language performance in Alzheimer’s disease and mild cognitive impairment: a comparative review. Journal of clinical and experimental neuropsychology, 30(5), 501–556. 10.1080/13803390701550128 [DOI] [PubMed] [Google Scholar]
  49. Teipel S, Ehlers I, Erbe A, Holzmann C, Lau E, Hauenstein K, & Berger C (2015). Structural Connectivity Changes Underlying Altered Working Memory Networks in Mild Cognitive Impairment: A Three-Way Image Fusion Analysis. Journal of neuroimaging : official journal of the American Society of Neuroimaging, 25(4), 634–642. 10.1111/jon.12178 [DOI] [PubMed] [Google Scholar]
  50. Themistocleous C, Eckerström M, & Kokkinakis D (2020). Voice quality and speech fluency distinguish individuals with Mild Cognitive Impairment from Healthy Controls. PloS one, 15(7), e0236009. 10.1371/journal.pone.0236009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Venneri A, McGeown WJ, Hietanen HM, Guerrini C, Ellis AW, & Shanks MF (2008). The anatomical bases of semantic retrieval deficits in early Alzheimer’s disease. Neuropsychologia, 46(2), 497–510. 10.1016/j.neuropsychologia.2007.08.026 [DOI] [PubMed] [Google Scholar]
  52. Vonk JM, Geerlings MI, Avila-Rieger J, Qian CL, Schupf N, Mayeux R, ... & Manly J (2022). Semantic item-level metrics relate to future memory decline beyond existing cognitive tests in older adults without dementia. [DOI] [PMC free article] [PubMed]
  53. Yeung A, Iaboni A, Rochon E, Lavoie M, Santiago C, Yancheva M, Novikova J, Xu M, Robin J, Kaufman LD, & Mostafa F (2021). Correlating natural language processing and automated speech analysis with clinician assessment to quantify speech-language changes in mild cognitive impairment and Alzheimer’s dementia. Alzheimer’s research & therapy, 13(1), 109. 10.1186/s13195-021-00848-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Zannino GD, Caltagirone C, & Carlesimo GA (2015). The contribution of neurodegenerative diseases to the modelling of semantic memory: A new proposal and a review of the literature. Neuropsychologia, 75, 274–290. 10.1016/j.neuropsychologia.2015.06.023 [DOI] [PubMed] [Google Scholar]

RESOURCES