What Drives Task Performance in Animal Fluency in Individuals Without Dementia? The SMART-MR Study

Abstract

Purpose:

In this study, we aim to understand whether and how performance in animal fluency (i.e., total correct word count) relates to linguistic levels and/or executive functions by looking at sequence information and item-level metrics (i.e., clusters, switches, and word properties).

Method:

Seven hundred thirty-one Dutch-speaking individuals without dementia from the Second Manifestations of ARTerial disease-Magnetic Resonance study responded to an animal fluency task (120 s). We obtained cluster size and number of switches for the task, and eight different word properties for each correct word produced. We detected variables that determine total word count with random forests, and used conditional inference trees to assess points along the scales of such variables, at which total word count changes significantly.

Results:

Number of switches, average cluster size, lexical decision response times, word frequency, and concreteness determined total correct word count in animal fluency. People who produced more correct words produced more switches and bigger clusters. People who produced fewer words produced fewer switches and more frequent words.

Conclusions:

Concurrent with existing literature, individuals without dementia rely on language and executive functioning to produce words in animal fluency. The novelty of our work is that such results were shown based on a data-driven approach using sequence information and item-level metrics.

Supplemental Material:

https://doi.org/10.23641/asha.23713269

Animal fluency is a widely used task in cognitive research and neuropsychological assessments, (Rofes et al., 2017; Rosser & Hodges, 1994). Participants are given 1 or 2 min to produce as many words as possible belonging to the semantic category of animals (e.g., dog, cat, horse, and elephant). To score this task, the total number of words produced is counted, excluding repetitions, proper names, and words that do not belong to the category animals (Thiele et al., 2016). This method can discriminate cognitively healthy individuals (i.e., without neurological or psychological difficulties) from people with depression, stroke, brain tumor, dementia, and brain infection (Ehlen et al., 2014; Faroqi-Shah & Milman, 2018; Fossati et al., 2003; Gaspers et al., 2012). This total correct score has also been used to measure cognitive differences linked to healthy aging (Beker et al., 2019), neurodegeneration (Marczinski & Kertesz, 2006), sleep disorders (Ma et al., 2020), and so forth.

Animal fluency is a “hybrid” task (Gordon et al., 2018; Shao et al., 2014; Whiteside et al., 2016), as it requires word retrieval and speed of lexical access to produce as many animals as possible, as well as executive functions to meet specific constraints when retrieving the words (use animals, do not repeat words, and avoid proper nouns). A debated question is whether and how animal fluency performance (i.e., total correct word count) relates to specific linguistic levels (semantics, phonological output lexicon, and phonological assembly) and/or executive functions (inhibition, mental set shifting, updating, and monitoring). This question corresponds to the problem of “task impurity,” which is used to stress that tasks do not necessarily make demands of a sole cognitive skill (Rabbitt, 1997). To answer this question, four different approaches have been employed in previous literature:

Comparisons Between Animal Fluency and Letter Fluency

In animal fluency, participants exhaust lexical entries belonging to the same semantic category, sharing “animal” features (Capitani et al., 1999). In contrast, in letter fluency, participants produce words that start with the same letter of the alphabet (e.g., F, A, and S); therefore, words do not necessarily share (many) semantic features (e.g., frog, flask, and free). As such, animal fluency may require more semantic input than letter fluency (Henry & Crawford, 2004). However, this reasoning does not imply that animal fluency does not require selecting entries in the phonological output lexicon (Rofes et al., 2020, 2021).

Associations Between Animal Fluency and Neuroimaging Data

Animal fluency recruits a large cortico-subcortical network with brain regions that overlap with letter fluency and that may be differently involved during task performance (e.g., Ahn et al., 2022; Biesbroek et al., 2021; Keser et al., 2020; Roberson et al., 2020). In terms of cortical structures, as measured in functional and structural MRI studies, letter fluency is typically reported as being mediated by frontal regions, whereas category fluency is additionally mediated by temporal regions (e.g., Gourovitch et al., 2000). For example, in whole-brain analyses of cortical thickness in cognitively normal adults, Vonk, Rizvi, et al. (2019) showed an association of letter fluency with primarily frontal regions and category fluency with both frontal and temporal–parietal regions. These differential cortical signatures in healthy adults confirm results from voxel-based lesion symptom mapping that associated frontal cortical damage with letter fluency and temporal cortical damage with category fluency (Baldo et al., 2006). Moreover, in a lesion-symptom mapping and structural disconnection study with a large number of people early after stroke (Biesbroek et al., 2021), further reliance of medial and posterior lateral structures and a unique contribution of the left pars triangularis for animal fluency (vs. letter fluency) was reported. Biesbroek et al. also discuss a greater involvement of animal fluency in lexical-semantic processes compared with phonological processes. Keser et al. (2020) reported associations between frontal lobe white matter measures in adults with multiple sclerosis and letter fluency but not with animal fluency. Hence, letter fluency may relate further to cognitive processes (e.g., executive functions) requiring the frontal lobe than animal fluency. Ahn et al. (2022) studied total scores, switches, and clusters during animal fluency in people with mild cognitive impairment (MCI) and people with Alzheimer's disease. Positron emission tomography results indicate circumscribed areas for switches and clusters, which may suggest that these item-level measures offer a more precise assessment of cognitive functions than the total score of animal fluency.

Associations Between Animal Fluency and Other Cognitive Tests

Associations between the total number of words and scores from other cognitive tests measuring similar cognitive/linguistic factors (solving mathematical problems, memorizing lists of words, matching a word with a set of possible descriptions, naming, and lexical decision) have shown that total number of words in animal fluency relates to other tests measuring attention (Amunts et al., 2021), cognitive flexibility (Paula et al., 2015), speed of lexical access (Gordon et al., 2018), vocabulary size (Whiteside et al., 2016), as well as working memory and inhibition (Amunts et al., 2021; Kavé & Sapir-Yogev, 2020; Shao et al., 2014; Unsworth et al., 2011; Whiteside et al., 2016). However, cognitive tests typically require more than one isolated cognitive/language function (Whitworth et al., 2014). For example, a language task such as written lexical decision requires the orthographic input lexicon—a store where participants “check” whether the string of letters they are given corresponds to an existing word or not. However, written lexical decision also requires letter recognition (i.e., to read the word or nonword) and access to semantics (since we normally read for meaning). Therefore, associations between lexical decision scores and animal fluency scores may not necessarily reflect only the phonological output lexicon (a store of spoken word forms) but also other linguistic levels, including semantics.

Associations Between Animal Fluency and Sequence Information/Item-Level Metrics

Linguistic variables (item-level metrics) and sequence information extracted from the words uttered by the participant can be associated with the total score, for example, cluster size, number of switches, and word properties/psycholinguistic variables of the fluency task (Mayr, 2002; Rofes et al., 2019; Thiele et al., 2016; Troyer et al., 1997).

The study of clusters and switches relies on the fact that participants produce words that are grouped into subcategories, presumably to support an internal search strategy (Abwender et al., 2001; Troyer et al., 1997). Regarding animal fluency, common subcategories include living environment (e.g., African animals), human use (e.g., farm animals), and taxonomy (e.g., bovine, feline). Clusters refer to words that belong to the same semantic family and that can be grouped under one such subcategory (Troyer et al., 1997). Cluster size has been related clinically to semantic memory impairment (Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998; Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998) and theoretically to the semantic system and/or the phonological output lexicon (Whitworth et al., 2014). Switches represent instances where participants move from one cluster or subcategory to another. For example, the series “lion, tiger, cow, sheep, horse, donkey” contains one switch, when the subcategory felines (i.e., lion and tiger) is replaced by farm animals (i.e., cow, sheep, horse, and donkey). ¹ Number of switches has been related to different aspects of executive functioning including information updating, monitoring, and set shifting (Gustavson et al., 2019; Miyake et al., 2000; Rofes et al., 2020).

Properties of words (frequency, imageability, and age of acquisition) can shed light on linguistic levels (i.e., semantics, phonological output lexicon, and phonological assembly) that are important for animal fluency performance (Forbes-Mckay et al., 2005; Marczinski & Kertesz, 2006; Vita et al., 2014). Word properties have been used to describe and classify the underlying linguistic impairment of individuals with brain damage (Nickels & Howard, 1994; Vonk, Jonkers, et al., 2019). Most importantly, issues with specific word properties have been linked to specific parts of the language system (Alyahya et al., 2020; Shallice et al., 2000; Whitworth et al., 2014; cf. Cutler, 1981). Word properties such as concreteness, familiarity, and imageability relate to semantic impairments (Nickels & Howard, 1994, 1995), whereas issues with length in graphemes and orthographic/phonologic neighborhood reflect problems in the phonological output lexicon or the phonological assembly (Caramazza et al., 1986; Shallice et al., 2000). Two other word properties, namely, age of acquisition and frequency, have been argued to reflect issues in either the semantic system (Brysbaert et al., 2000; Ralph et al., 1998; Steyvers & Tenenbaum, 2005) or the phonological output lexicon (Brown & Watson, 1987; Ellis & Lambon Ralph, 2000; Gvion & Friedmann, 2016).

Our team recently used a method to analyze several item-level metrics and sequential information that is relatively unaffected by variables that correlate with one another. This approach is relevant because, for example, frequency has been shown to correlate with the age of acquisition (e.g., Ellis & Morrison, 1998) and frequency with length (e.g., Hauk & Pulvermüller, 2004), and this in turn makes it difficult to provide clear results regarding the linguistic processes affected or relevant to perform different tasks. In our results, we showed that, among several item-level metrics, number of switches was the most strongly related variable to the total number of words in people with Alzheimer's disease and in people with HIV (Rofes et al., 2020, 2021). In addition, age of acquisition in people with Alzheimer's disease and word frequency in people with HIV determined the total number of words produced on animal fluency. Similar to Shao et al. (2014), these results stress the “hybrid” nature of animal fluency. Given similar analyses, a question that remains open is whether and how performance in animal fluency relates to linguistic levels and/or executive functions in individuals without dementia. This new study is valuable provided that relatively little work has examined verbal fluency in adult Dutch speakers without apparent neurological or psychiatric disorders (e.g., Van Der Elst et al., 2006). Also, it stresses patterns of performance during animal fluency in older adults without dementia, facilitating future comparison studies to detect deviant patterns in people with notable language impairments. The novelty of this work also lies in the use of a data-driven approach to looking at sequence information and item-level metrics in a relatively large number of participants.

Aims and Hypotheses

We assessed if and how sequence information and item-level metrics of animal fluency reveal the linguistic levels (i.e., semantics, phonological output lexicon, and phonological assembly) and executive processes (i.e., updating, shifting, and monitoring) needed to perform this task in a sample of people without dementia. We investigated (a) the independent associations of 10 animal fluency metrics with the total number of correct words in individuals without dementia, (b) which sequence information or item-level metrics determined the total number of correct words, and (c) how they interconnect in their relation to the total number of words.

We hypothesized that sequence information and item-level metrics related to both language and executive functions would determine the total number of words. Specific to executive functions, we expected number of switches to be a strong determinant of the total number (Abwender et al., 2001; Rofes et al., 2020, 2021). Specific to language function, we expected frequency and age of acquisition to be related to total number of words (Forbes-Mckay et al., 2005; Rofes et al., 2020, 2021; Vonk, Jonkers, et al., 2019). Regarding the interconnection of metrics that determine the total number of words, we expected number of switches and age of acquisition to be the strongest determinants of total number of words (Rofes et al., 2020). However, we had some reservations provided that previous analyses were only conducted on data from neurological populations.

Method

Participants

We included cross-sectional data of 731 Dutch-speaking adults without dementia (see Table 1). The data belong to the SMART-MR study (Second Manifestations of ARTerial disease-Magnetic Resonance). This is a prospective cohort study that includes people at higher risk of developing dementia due to atherosclerotic disease (Geerlings et al., 2010). Individuals underwent up to three visits: baseline (n = 1,309), after approximately 4 years (n = 754; Retention Rate Visits 1 and 2 = 57.6%), and after approximately 12 years (n = 329; Retention Rate Visits 2 and 3 = 43.6%; recruitment and procedures described elsewhere, Geerlings et al., 2010). Participants (N = 1309) were 20.3% women, partially reflecting differences in cardiovascular disease between men and women. The participants' ethnicity is approximately 97% White, 1.5% Southeast Asian, 1% Black, and 0.5% Northern African or Middle Eastern. We used data from the second visit, because animal fluency was not administered as part of the baseline visit; participant selection is detailed in the flowchart in Figure 1. Among the 754 participants, we excluded 18 participants that had missing animal fluency data and five participants that had missing item-level fluency data. We did not exclude 38 participants that had a diagnosis of MCI as per the Petersen et al. (2014) criteria, and nine participants did not have enough data available to exclude this diagnosis. We did not exclude these participants because (a) a research diagnosis of MCI has been shown to be unstable over time with high rates of reversion to normal at a later follow-up (Angevaare et al., 2022) and (b) our goal was to investigate individuals without dementia; MCI is a risk factor for dementia but does not define a preclinical stage (Canevelli et al., 2016). Please see Supplemental Material S1 for results excluding people with MCI. Written informed consent was obtained from all participants according to the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act and the policies of the Medical Ethics Research Committee of the UMCU.

Table 1.

Sample characteristics (n = 731).

Demographic	Value
Age	61.7 (9.5), 31–83
Males/females	601/130 (82%/17%)
Education: college/university	186 (25%)
High school	475 (65%)
< High school	65 (9%)
Unknown	5 (1%)
Animal fluency
Total number of words	30 (8.4), 3–61
Cluster size	2.7 (1.2), 1–23
Number of switches	10.9 (3.9), 0–23
Lexical decision response times	534.0 (11.7), 497–585
Age of acquisition	6.6 (0.8), 4–12
Concreteness	4.8 (0.1), 4.4–4.9
Frequency	2.2 (0.3), 0.5–3.1
Orthographic neighborhood	7.2 (1.4), 2.0–13.3
Phonologic neighborhood	8.9 (2.0), 2.7–17.8
Word length in phonemes	4.7 (0.5), 3–8.4
Word length in graphemes	5.4 (0.5), 3.7–9.7

Note. (Standard deviation) and range.

Figure 1.

Participant inclusion. SMART-MR = Second Manifestations of ARTerial disease-Magnetic Resonance.

Task and Scoring

Participants were given 120 s to say as many words as possible belonging to the category animals, starting with any letter of the alphabet (e.g., hond, kat [dog, cat]). The scoring consisted of the following steps: First, the words produced by each participant were written in an article scoresheet by the examiner; for this study, these words were entered by research assistants into a digital database. Second, the words counting toward the total score were counted by disregarding intrusions (i.e., words that are not animals), repetitions, perseverations (repetition of an answer), proper names (e.g., John and Mary), and foreign words. Third, cluster size and number of switches were extracted with Semantic Network and Fluency Utility (SNAFU; Zemla et al., 2020) using a Dutch adaptation of the animal scheme (github.com/jmjvonk). Finally, eight different word properties were extracted for each word counting toward the total score: age of acquisition, concreteness, word frequency, orthographic neighborhood, phonologic neighborhood, lexical decision response times, word length in phonemes, and word length in graphemes.

For the metrics related to the generated sequence of words, clusters were defined based on categorization in the seminal work by Troyer et al. (1997), an expanded categorization by Hills et al. (2012), and further expansion of this categorization by Zemla et al. (2020) for the SNAFU application. The following animal subcategories were considered: African, Australian, Arctic/Far North, farm, North American, water, beasts of burden, used for their fur, pets, birds, bovines, canines, deer, felines, fish, forest, insects, insectivores, primates, rabbits, reptiles/amphibians, rodents, weasels, marsupials, European, arachnids, South American, worms, mythical creatures, unicellular organisms, and zoo. SNAFU provides an estimate of an inherently ambiguous metric, as subcategories may overlap and it is not possible to assess which subcategories participants may have focused on during the task. Subsequently, SNAFU calculated the average cluster size per participant. SNAFU also calculated the number of switches between clusters per participant, by counting the instances where the category label of a newly named word differed from the previous word. For example, the sequence “horse, cow, pig, elephant, lion, tiger” contains two clusters: farm animals (i.e., horse, cow, and pig) and African animals (i.e., elephant, lion, and tiger), and the size of each of the clusters is three words.

For the metrics related to word properties, age of acquisition ratings was extracted from Brysbaert et al. (2014). This database contains the scores of 30,000 Dutch words obtained by asking cognitively healthy adults to assign an approximate age at which they learned a specific word. Concreteness ratings were also obtained from Brysbaert et al. (2014); in this case, cognitively healthy participants were asked to assign the level of concreteness of a word on a 5-point scale (1 = very abstract/language-based, 2 = more abstract than concrete [language component is more important than experience component], 3 = neutral, 4 = more concrete than abstract, and 5 = very concrete/experience-based). Word frequency ratings were obtained from SUBTLEX-NL (Keuleers et al., 2010); we used logarithmic frequency values (inherent to SUBTLEX-NL) of a corpus of subtitles of Dutch TV shows and movies that include over 43 million words. Orthographic and phonologic neighborhood densities were obtained from CLEARPOND (Marian et al., 2012). The neighborhood density values of this database were calculated by counting the number of words (based on the corpus of SUBTLEX-NL) that differed from each target word by adding, deleting, or substituting one grapheme/phoneme. For example, the word konijn (i.e., rabbit) has an orthographic neighborhood density of two, given the words tonijn and komijn (i.e., tuna and cumin). Word length in phonemes was also retrieved from CLEARPOND (Marian et al., 2012). Lexical decision response times for each word were extracted from the Dutch Lexicon Project 2 (Brysbaert et al., 2016). These correspond to the average time that cognitively healthy individuals needed to respond whether a word was a real word or not in a lexical decision task. Finally, word length in graphemes was calculated using the formula “LEN=” in Excel.

Across the whole data set of animal fluency responses, the number of missing values for age of acquisition and concreteness represented 2.9 and 3.6% of the words, respectively. For orthographic/phonological neighborhood density, 14.6% of the words had missing values. To minimize such missing values, we first checked for synonyms, as indicated in a Dutch dictionary (van Sterkenburg, 1991). For example, the word nachtvlinder (i.e., moth; “butterfly of the night”) was replaced with mot (i.e., moth), since both words represent the same animal. However, houtduif (wood pigeon) was not replaced with duif (pigeon), because “wood pigeon” is a specific type of “pigeon.” Therefore, houtduif and other animals with no suitable substitute were imputed: For age of acquisition, we assigned the value of 12, because a child's vocabulary typically reaches a plateau at that age (Beitchman et al., 2008). For concreteness, we imputed with the participant's mean score. For word frequency, we assigned the logarithm (log) of an occurrence of 0.5 times in the > 43-million-word database. For orthographic/phonological neighborhood density, we imputed the missing words directly with the participant's mean because neighborhood densities would become unrealistic if we were to replace missing values with substitute words. For length in phonemes, we measured manually.

Statistical Analyses

We used random forests for regression and “variable selection” (Breiman, 2001). That is, to choose which variables function as better determinants (or predictors) of the total number of words in animal fluency in this sample. Random forests are particularly suited to this data set, as they include a relatively large number of predictive variables (n = 10), some of which may be correlated. For example, other authors report correlations between values of concreteness, word frequency, and age of acquisition (Brown & Watson, 1987; Rochford & Williams, 1965).

Additionally, conditional inference trees (Strobl et al., 2008) can identify points along the scale of a predictive variable where the values of the dependent (or response) measure change significantly (i.e., split points). The end result is a tree-like representation, with nodes representing split points for predictive variables (age of acquisition) that are related to the dependent variable (total number of words). This approach is robust to outliers (Loureiro et al., 2004) and has previously been used by our team. For example, we reported that people with Alzheimer's disease who produced words with a mean age of acquisition greater than 4.64 (on a scale of 1–10) produced more words in animal fluency than people with Alzheimer's disease who produced words with a mean age of acquisition equal to or smaller than 4.64 (Rofes et al., 2020).

The analyses were structured as follows: First, we generated a random forest with unbiased conditional inference trees (Strobl et al., 2008) by using the cforest function in R (Hothorn et al., 2017). Second, we used the “conditional permutation importance” (CPI) metric, calculated by the varimp function (Hothorn et al., 2017), which informs variable selection by indicating the extent to which a predictive variable in the fitted random forest determines the dependent variable. A predictive variable is ranked high in CPI when removal of this variable from the model results in a decrease in model fit (Strobl et al., 2008). Third, we estimated model fit including only predictive variables that ranked high in CPI. To do that, we used leave-one-out cross-validation (Strobl et al., 2008)—a procedure in which the classifier is trained on a data set in which one data point (i.e., one participant) is omitted at a time. The estimated value of the observation with the omitted participant is then calculated and saved. This procedure was repeated for each data point. To finalize this step, we examined the relation between the observed values and the estimated values of the total number of words. Fourth, we evaluated the model fit, as measured by R², root-mean-squared error (RMSE), and mean absolute error (MAE). After the random forests, in the fifth step, we used conditional inference trees to examine split points for predictive variables that were ranked high in CPI, as indicated by the random forests. We restricted the maximum depth of the tree (maxdepth; i.e., the number of nodes along the longest path from root to leaf) to 2, to facilitate the interpretation of the results. Analyses were conducted in R Version i386 4.0.3 (R Core Team, 2020), and all code is available at github.com/jmjvonk).

Results

Descriptive Statistics

Sample characteristics, including summary statistics of the demographics (i.e., age, sex/gender, and education), total correct word count, cluster size, number of switches, and the eight word properties can be found in Table 1.

Item-Level Metrics Related to Total Score

The regression model with random forests to indicate variables ranking high in CPI explained 88% of the variance in the dependent measure total number of words (R² = 0.885; RMSE = 2.85; MAE = 1.70), after the leave-one-out cross-validation. The variables that accounted for most of the variance in the regression model (i.e., ranking high in CPI) were in order of higher to lower CPI: number of switches, average cluster size, lexical decision response times, word frequency, and concreteness. A second model, calculated only with the variables ranking high in CPI, predicted 79% of the variance (R² = .798; RMSE = 3.77; MAE = 2.69).

Using the variables ranking high in CPI in the random forest analysis, conditional inference trees identified split points among number of switches, average cluster size, and word frequency (R² = .82; RMSE = 3.55; MAE = 2.35). Note that all five high CPI variables were considered as split points, but only three variables were identified as split points in the conditional inference trees analyses. The split point at the highest node of the tree (Node 1, for number of switches) indicates that participants with a mean number of switches larger than 11 (n = 303, sum of Nodes 6 and 7) produced more words compared with participants with a mean number of switches equal to or smaller than 11 (n = 428, sum of Nodes 3 and 4; X² = 254.723, p = .0001). Two other split points were identified. To the right of the tree, among participants with a mean number of switches larger than 11 (Node 5, for average cluster size), the split point indicates that participants with average cluster sizes larger than 2.154 (n = 163, Node 7) produced more words compared with participants with average cluster sizes equal to or smaller than 2.154 (n = 140, Node 6; X² = 119.253, p = .0001). Finally, to the left of the tree, among participants with a mean number of switches smaller than 11 (Node 2, for word frequency), the split point indicates that participants with mean word frequency scores larger than 2.346 (n = 146, Node 4) produced fewer words compared with participants with word frequency scores equal to or smaller than 2.346 (n = 282, Node 3; X² = 142.945, p = .0001; see Figure 2).

Figure 2.

Conditional inference tree for animal fluency. Conditional inference tree with number of switches (num_cluster_switches), average cluster size (avg_cluster_size), and word frequency (frequencyanimal_avg) as relevant variables. Conditional inference tree representing how variables interact. Each circle/box represents a node. The number of each node is written on top of the circle. Nodes represented with circles show variables that are significantly associated with different distributions of the value of the dependent variable (i.e., total number of correct words), and include a p value. Below each circle, the values indicate a point in each variable where individuals split into those groups (more than 11 for number of switches). The nodes represented by boxes are the terminal nodes, which describe a subsection of the sample that corresponds to that split in the data and provides a plot representing the distribution of the data for each subsection of the sample (the box in Node 7 includes 163 participants producing an average of 38 words in animal fluency).

Discussion

Using a data-driven analysis of animal fluency data, we investigated whether performance as measured by the total correct word count relates to sequence information and item-level metrics that represent linguistic levels and/or executive functions, including number of switches, average cluster size, and eight different word properties. The results indicate that number of switches, average cluster size, lexical decision response times, word frequency, and concreteness predict the total number of words in animal fluency in individuals without dementia. These five factors accounted for 79% of the variance (maximum 100%). Overall, the results are in agreement with previous literature, in particular, with the fact that animal fluency tasks require both language and executive functions (Gordon et al., 2018; Shao et al., 2014; Whiteside et al., 2016). The discussion will focus on the number of switches, average cluster size, and word frequency, as these variables were also identified in the conditional inference trees.

Similar to two other recent studies (Rofes et al., 2020, 2022), number of switches was the most important variable (i.e., highest ranking in CPI) to determine the total number of words. This variable corresponds to executive functioning (Gustavson et al., 2019; Miyake et al., 2000). In our view, switching mostly relies on information updating and monitoring (Rofes et al., 2020). This is because switches reflect the renewing of criteria to search for words, while keeping track of the words produced, and adhering to the specific instructions required in the task (i.e., produce as many animals as possible in 1 min, do not repeat words, do not produce proper names). It is unlikely that switching relies on the inhibition component in the Miyake model, because it does not reflect controlled suppression of responses, such as the type of function needed in picture word interference paradigms, where participants name an object with a semantically related word written on top (Shao et al., 2014) or in an arrow flanker task (Unsworth et al., 2011). Although it may rely on the shifting component in the Miyake model, we consider this less likely because switches reflect the exhaustion in the generation of items within one subcategory; hence, the renewing of the criteria to search for words may occur without an active strategy to find more words.

Further work seems needed, as shifting measures and verbal fluency have not been directly evaluated in this or other work. In fact, shifting may be involved if an active search strategy is used. Participants may monitor the number of words they produce, and, when the number of words is below a certain self-determined threshold, they may actively switch their attentional focus to a new set of subcategories. To assess whether an active strategy is used (vs. a renewal of the search criteria), one possibility is to correlate switches with tasks that require shifting in an active manner. One of such tasks is the Wisconsin Card Sorting Test (WCST; Kimberg et al., 1997). However, in the WCST, the shifting strategy is induced by the experimenter, while shifting is driven by the participant (or occurs spontaneously, as we argue) in animal fluency. A second option is to study how switches occur. For example, participants may start a new cluster with words unrelated to the previous cluster (in “dog, cat, elephant, hippopotamus, lion,” elephant starts a new category “African animals,” but “dog, cat” are at least not prototypical “African animals”) or with words that are related/primed (in “hippopotamus, elephant, flamingo, pigeon, dove,” flamingo starts a new subcategory “birds,” but flamingo is also an “African Animal”). Identifying whether participants use a prominent strategy to switch between clusters could indicate an active strategy. To do so, the scores of people who are asked to use clusters during animal fluency could be compared against a sample of people who receive no explicit training. Finally, we cannot disregard the potential relation between switches and language, since switches are dependent on the ability of the participant to retrieve words within a subcategory (Mayr, 2002). Also, we cannot rule out other cognitive contributors, given that we used a single task (Rabbit, 1997).

The cutoff value of switches was 11 in this study, as indicated by the conditional inference trees. This value does not match the cutoff values of two of our previous studies: Rofes et al., 2020 (5.8 words) and Rofes et al., 2022 (24 words). This discrepancy might have occurred because the former studies looked at data sets from people with clinically diagnosed neurological/language impairments: people with Alzheimer's disease and people with HIV. The time allotted for animal fluency in each study could also have played a role. However, in Rofes et al. (2022), we also used a 2-min animal fluency task and the cutoff value was more than double as found in this study. Importantly, the direction of the effect was the same across the three studies. Namely, people that produced more switches (regardless of the exact number) also produced more correct words in animal fluency. This consistent finding suggests that the ability to renew the criterion to search for words within the same semantic category is a powerful strategy to produce more words in animal fluency. In fact, in the current data set, those individuals that used such a strategy (i.e., produced more than 11 switches) produced twice the average number of words (n = 40 words) than those that did not (n = 20 words).

Participants who produced the highest number of words also produced clusters that were on average larger than 2.154 words (n = 163, Node 7). The observation that cluster size determines the total number of words indicates that animal fluency requires language information, be it in the semantic system or the phonological output lexicon (Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998; Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998; Whitworth et al., 2014). Interestingly, this split point in the size of the cluster represents the smallest type of cluster that there is, that is, a cluster containing two words (Troyer et al., 1997; Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998; Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998). Therefore, participants that produced most words produced a larger number of clusters of any size, rather than words that cannot be semantically grouped.

Participants who produced the least number of words also produced fewer switches and words of higher frequency (mean log frequency = 2.346). Word frequency can be associated with the phonological output lexicon, since words of high frequency are more likely to be accessed, as they require less activation to be retrieved than low-frequency words (Brown & Watson, 1987; Ellis & Lambon Ralph, 2000; Nickels & Howard, 1994). However, word frequency has also been related to the semantic system, as people with semantic dementia (i.e., a neurodegenerative disease) may fare worse with low- than high-frequency words (Ralph et al., 1998; Vonk, Jonkers, et al., 2019). Since frequency was the only word property detected in the conditional inference trees, by looking only at sequence information and item metrics, it seems difficult to make a clear-cut distinction between the role of semantics and/or the phonological output lexicon in animal fluency.

Strengths of this study include using a data-driven approach, a relatively large number of metrics extracted from the task itself (n = 10), and a large number of participants. This approach differs from studies using external tests or correlational inferences from data of people with brain damage and/or neuroimaging data (Amunts et al., 2021; Roberson et al., 2020; Shao et al., 2014; Vonk, Rizvi, et al., 2019). A limitation of our study is that our sample included people with MCI and people with subjective complaints. We decided not to exclude these people because these categories do not represent people with dementia, the types of cognitive impairments leading to being labeled as MCI are highly variable (Petersen et al., 2014), and because in prospective cohort studies, it is not unusual that people diagnosed with MCI can be labeled as normal at a follow-up visit (Angevaare et al., 2022; Canavelli et al., 2016). That said, as it can be seen in Supplemental Material S1, the exclusion of participant diagnosed with MCI in our sample led to similar results, with the exception that word frequency did not explain total number of words. Another limitation of our study is that the data were collected by different professionals (nurses and research assistants) over a period of 3 years (i.e., 2006–2009). This creates an issue of interrater reliability, for example, in the accuracy with which the raw data were entered in the database. Nonetheless, these two issues may be minimized provided the relatively large sample size of this study.

Regarding our approach, one issue is that the sequence information and the item-level metrics we used rely on the number of correct words produced by the participant. Although this is common practice (Rofes et al., 2019; Thiele et al., 2016; Troyer et al., 1997; Vonk, Flores, et al., 2019; Vonk et al., 2021), it is unknown whether a minimum number of words may be necessary to provide reliable results for these kinds of analyses. This concern seems relatively minor for studies of cognitively healthy individuals who can produce a normal number of words. However, it may be different for neurological populations, particularly if they produce a low number of correct words. Further studies could estimate the minimum number of words necessary to provide reliable results for measures such as the ones we considered in this study (Ossewaarde et al., 2020).

In addition, other analyses of the total number of words (including errors such as repetitions or words belonging to another category) or studying unconstrained fluency (where participants can produce words of any semantic category; Beausoleil et al., 2003) may provide larger number of words to analyze than using the correct words, and maybe more insight into the word properties produced (cf. Gaspers et al., 2012, for a study where total number of correct words is regarded as a superior measure than total number of words). For example, a participant producing on average animals of high frequency may also produce low-frequency words belonging to a non-animal category. Furthermore, the fact that animal fluency mainly relies on the use of concrete nouns (perhaps with the exception of potential mythological creatures) limits the utility of this variable as a predictor. Also, age of acquisition ratings from adult reports may indicate the order in which the words were acquired, rather than the actual age at which words were acquired (cf. De Moor et al., 2000). To some extent, this lowers the variability in the ratings, particularly for words learned earlier in life. Finally, the number of values we needed to impute for neighborhood density scores was particularly high (14.6%).

Another limitation is that we did not measure whether participants that produced more words came back to the same cluster after one or more switches. For example, a participant could start with farm animals, continue with a cluster of worms, and go back to farm animals. These analyses could shed further light on the nature of switches. For example, if switches are an indicator of executive functions, we may expect (smaller) clusters of the same kind to repeat. However, if they are an indicator of linguistic levels, then we may expect clusters of the same kind not to repeat as, in essence, switches would indicate the definite exhaustion of the lexical search of a given semantic subcategory, rather than a strategy to renew the criteria when it is more difficult to find words. Of course, if we only count correct words, the extent to which a cluster can repeat should be dependent on the size of the cluster and, on average, we could expect knowledge of certain semantic subcategories to be higher (and therefore, more readily available) in the general population compared with others (i.e., farm animals vs. worms).

Future research could contrast item-level information of animal fluency with that of other types of category fluency tasks (e.g., fruits, vegetables, and tools; Capitani et al., 1999) or with item-level metrics of letter fluency, complex fluency (e.g., possible jobs, consequences; Butler, 1993; Gordon et al., 2018; Guilford, 1967), verb fluency, and unconstrained fluency. These analyses may help to assess whether the relationship of word properties with the total number of words differs as a function of the type of fluency task. For example, an association of word frequency with the total number of words produced in different category fluencies but not in letter fluency would suggest that word frequency reflects semantic processing rather than the phonological output lexicon. To further investigate these performance patterns, additional experimental work should be performed that focuses on revealing the strategies that optimize performance, for example, a controlled shift-strategy intervention design. In addition, the same analyses on data of two different populations (language impaired and not) matched for demographic variables may be an option to corroborate the current results and to stress their validity in terms of pointing to deviant language patterns.

In conclusion, similar to other studies (Gordon et al., 2018; Shao et al., 2014; Whiteside et al., 2016), individuals without dementia rely on language and executive functioning to produce words in animal fluency. The novelty of our work is that such results can be shown using sequence information and item-level metrics and a data-driven approach, in a sample of individuals without dementia. This work highlights the specific linguistic levels and executive functions that determine performance in individuals without dementia, which can be used to define the extent to which performance deviates from these results in populations with dementia and other communication disorders. As such, these results may aid in understanding and identifying language impairments.

Data Availability Statement

For use of SMART-MR data, a request has to be made for UCC-SMART data (https://www.umcutrecht.nl/en/utrecht-cardiovascular-cohort). A proposal needs to be submitted and approved, after which arrangements can be made to get access to the data. For more information, e-mail uccdatarequest@umcutrecht.nl. The data are not publicly available due to privacy and ethical restrictions.

Funding Statement

The authors report no conflict of interest concerning the materials, methods, or findings specified in this article. Jet M. J. Vonk was supported by the National Institute on Aging under Award K99AG066934, NWO/ZonMw under Veni Grant Project Number 09150161810017, and by Alzheimer Nederland under Fellowship WE.15-2018-05. Adrià Rofes was supported by NWO under SSH XS Grant Project Number 406.XS.01.050.