The Victoria Stroop Test: Normative Data in Quebec-French Adults and Elderly

Abstract

Objective: Despite the widespread use of the Victoria Stroop Test (VST; Regard, 1981) in clinical and research settings, information regarding the impact of sociodemographic variables on test performance in Quebec-French adults and elderly people is still nonexistent. Thus, this study aimed to establish normative data for error scores and completion time on all test trials (Dot, Word, and Interference) taking into account the impact of age, education, and sex on test performance.

Method: The sample consisted of 646 community-dwelling and healthy Quebec-French individuals aged between 47 and 87 years.

Results: Regression analyses indicated that age was associated with completion time and error scores on all trials. The association was also positive for low and high interference conditions. Education was associated with completion time on Word and Interference trials, and with both interference scores. Finally, sex was associated with completion time on all trials, with women being consistently faster than men. Equations to calculate Z scores and percentiles are presented.

Conclusions: Norms for the VST will ease interpretation of executive functioning in Quebec-French adults and elderly and favor accurate discrimination between normal and pathological cognitive states.

Introduction

The Stroop Test (Stroop, 1935) is among the most frequently used tools to assess several components of executive functions, such as cognitive flexibility, selective attention, cognitive inhibition, and information processing speed. This task measures the ability with which an individual can maintain a goal and inhibit an automatic, overlearned response, in favor of a less familiar one (Strauss, Sherman, & Spreen, 2006). Several versions of the Stroop test have been developed (e.g., Dodrill, 1978; Golden, 1978; Graf, Uttl, & Tuokko, 1995), with variations in the administration procedure, the number of trials, the color and number of items, the method of scoring, and the normative data calculation (Bayard, Erkes, & Moroni, 2011; Strauss et al., 2006; Troyer, Leach, & Strauss, 2006). However, all versions share the same paradigm, which involves a control task that requires participants to name the colors of non-word items (e.g., dots, Xs) and an interference task that requires participants to inhibit reading a color word and rather name the color of the ink in which it is written (e.g., say green for the word “red” printed in green). The increase in time taken to perform the latter task compared with the control task is known as the Stroop interference effect (Stroop, 1935).

The assessment of executive functions is a critical component of the neuropsychological evaluation of older adults. In fact, normal aging and neuropsychological conditions, such as mild cognitive impairment and Alzheimer's disease, have been found to weaken executive functions, and more particularly inhibitory processes (Belleville, Chertkow, & Gauthier, 2007; Belleville, Rouleau, & Van der Linden, 2006). The Victoria Stroop Test (VST; Regard, 1981), a briefer version than the original Stroop Test, is particularly appropriate for use in older individuals because of its short administration time (Troyer et al., 2006). The VST is sensitive to several neuropsychological states, namely frontal lobe damage, mild cognitive impairment and dementia (e.g., Bondi et al., 2002; Joubert et al., 2010; Nathan, Wilkinson, Stammers, & Low, 2001).

In the VST, three cards of 24 items, colored in blue, green, yellow or red and distributed in six rows of four items, are presented to participants who are asked to name as quickly as possible the color of dots (Dot condition—Card 1), the color of the ink in which neutral words are printed (Word condition—Card 2), and the color of the ink in which color names are printed (Interference condition—Card 3). For each condition, the completion time and the number of errors are compiled, and interferences scores are derived by calculating the ratio between the time required to name the color of the ink in the Word and the time required to name the color of dots in the Dot conditions (low interference), and the ratio between the time required to name the colors in the Interference and the Dot conditions (high interference) (Strauss et al., 2006).

There is growing evidence for cultural differences regarding executive functions and attention processes (e.g., Hedden, Ketay, Aron, Markus, & Gabrieli, 2008; Hedden et al., 2002; Nisbett & Masuda, 2003; Nisbett, Peng, Choi, & Norenzayan, 2001; Park & Gutchess, 2006; Park, Nisbett, & Hedden, 1999). Interestingly, one study demonstrated that Chinese preschoolers performed significantly better than their American counterparts on all measures of executive functioning (Sabbagh, Xu, Carlson, Moses, & Lee, 2006). In another study, African-American elderly were found to perform at a lower level than Caucasian individuals on the Stroop test (Moering, Schinka, Mortimer, & Graves, 2004). In this respect, several culture-specific normative studies were carried out for different measures of executive functions (e.g., Abe et al., 2004; Hashimoto et al., 2006; Moering et al., 2004; Steinberg, Bieliauskas, Smith, & Ivnik, 2005; Van der Elst, Van Boxtel, Van Breukelen, & Jolles, 2006). Moreover, in a recent study, the researchers suggested that local norms are more precise to identify cognitive difficulties in older adults when compared with non-culture-specific norms (Arsenault-Lapierre et al., 2011). To the best of our knowledge, only two normative studies have been published for the VST in adults and elderly people (Bayard et al., 2011; Troyer et al., 2006). These normative studies were developed using: (1) a sample of 244 healthy community-dwelling French adults aged between 50 and 94 years old from Montpellier and Lille (France; Bayard et al., 2011) and (2) a sample of 272 healthy and community-dwelling English-speaking adults aged between 18 and 94 from Ontario (Canada; Troyer et al., 2006). However, yet no normative study has been carried out in a Quebec-French population.

Although cultural differences in cognition do exist, sociodemographic variables, such as age, education level, and sex, have been shown to influence performance on neuropsychological tests (Mitrushina, Boone, & D'Elia, 1999; Mitrushina, Boone, Razani, & D'Elia, 2005). Most of the studies that were carried out for the Stroop test reported an age-related decline in performance (e.g., Amato et al., 2006; Ivnik, Malec, Smith, Tangalos, & Petersen, 1996; Moering et al., 2004; Van Boxtel, ten Tusscher, Metsemakers, Willems, & Jolles, 2001). More specifically, one study suggests that the speed-dependent variables, rather than the accuracy measures, are thoroughly modulated by demographic variables (Van der Elst et al., 2006). Consistent with these results, Bayard and colleagues (2011) have shown that the completion time of the VST was largely associated with age, while no correlation between age and interference scores was observed. However, Troyer and colleagues (2006) proposed that not only completion time but also error scores were associated with age. The influence of education and sex on the performance to the VST is less consistent. In fact, some authors have argued that Stroop performance was linked to education level (Moering et al., 2004; Steinberg et al., 2005; Van Boxtel et al., 2001; Van der Elst et al., 2006), while others did not obtain these results (Amato et al., 2006). Also, while women have been shown to perform better than men in some studies (e.g., Strickland, D'elia, James, & Stein, 1997; Van der Elst et al., 2006), sex differences were not significant in others (e.g., Anstey, Dain, Andrews, & Drobny, 2002; Ivnik et al., 1996). With respect to education, only one VST normative study found a greater performance in more educated people (Bayard et al., 2011) and no study found an association between sex and performance (Bayard et al., 2011, Troyer et al., 2006).

Despite the widespread use of the VST, normative data and information regarding the influence of sociodemographic factors on test performance in Quebec-French adults and elderly people is nonexistent. Consequently, this study aimed to establish normative data for Quebec-French adults and elderly people. The impact of age, education, and sex was examined for error scores and completion time for all trials, as well as for interference scores (low interference and high interference). The development of VST norms is necessary to allow precise interpretation of test results and ensure accurate discrimination between normal and pathological cognitive states. For this purpose, the French adaptation of the VST was used (Moroni & Bayard, 2009).

Methods

Participants

Researchers across the province of Quebec (Canada) were invited to share anonymized data from healthy volunteers who had completed the VST as part of other research studies approved by local Research Ethics Boards. Secondary data from those studies were used in this normative study. Participants originated from three laboratories in Montreal and Quebec City.

All participants included in the study were Caucasian, lived independently in the community, and reported French as their mother and usual tongue. All selected participants scored within normal range on cognitive screening instruments, that is the Mini-Mental State Examination (MMSE > 26; Folstein, Folstein, & McHugh, 1975) or the Montreal Cognitive Assessment (MoCA > 26; Nasreddine et al., 2005). Furthermore, subjects had no depression as assessed by the Geriatric Depression Scale (GDS < 11; Yesavage et al., 1983). Any participant who reported untreated medical conditions (e.g., diabetes, hypertension) that could interfere with normal cognitive aging, or a history of neurological disease, mental retardation, psychiatric illness, head injury, or drug abuse according to DSM-V criteria (American Psychiatric Association, 2013) were excluded from the study sample.

The final normative sample consisted of 646 community-dwelling adults and elderly people (383 women; 263 men), aged between 47 and 87 (mean age = 74.80 years; SD = 7.09), with an education level varying between 3 and 23 years (mean education level = 13.33 years; SD = 4.21). Compared with actual Quebec demographics, our sample had an overrepresentation of women and of highly educated individuals (Institut de la statistique du Québec, 2006).

Materials and Procedure

Trained professionals administered the French adaptation of the VST to all subjects (Moroni & Bayard, 2009). There were 24 items in each condition, described below. In the “Dot” condition, colored (blue, green, yellow, and red) dots were presented. In the “Word” condition, the words mais (but), pour (for), donc (thus), and quand (when) were written in one of the four color inks mentioned above. In the “Interference” condition, the words bleu (blue), vert (green), jaune (yellow), and rouge (red) were written in one of the three other colors (e.g., word green written in yellow ink). In each of the three conditions, participants were asked to name as quickly as possible the color of the dots (“Dot” condition), to name the color of the ink in which neutral words are written (“Word” condition), and to name the color of the ink in which color words are written (“Interference” condition). For each condition, the completion time and the number of errors were compiled. An interference effect was derived from this task by calculating the ratio between the time required to name the color of the ink in the “Word” condition and the time required to name the colors in the “Dot” condition (low interference), and the ratio between the time required to name the colors in the “Interference” and “Dot” conditions (high interference) (Strauss et al., 2006).

Statistical Analyses

First, to identify the variables influencing performance on the VST, a standard multiple regression analysis was performed for completion time on “Dot”, “Word,” and “Interference” conditions with age, sex, and education as predictors. Second, regression analyses were performed for Interference index scores, namely “Word/Dot for time” and “Interference/Dot for time”, with age, sex, and education as predictors. All variables were entered in the analyses as continuous variables except sex, which was coded as 0 for women and 1 for men. Visual and statistical analyses were conducted to verify the underlying assumptions of the regression model. Because of the skewed distribution of error scores in all conditions of the task, partial Spearman correlations were computed to determine the sociodemographic variables independently associated with error scores. Percentiles for each condition were generated and were stratified according to the significant correlates. For example, if only age was significantly correlated with performance, then percentiles were stratified by age only. All statistical analyses were performed using IBM SPSS Statistics software (version 21.0) with the alpha level set at 5%.

Results

Mean completion time and error scores for the normative sample on each condition are presented in Table 1.

Table 1.

Mean completion time and error scores for the normative sample on Dot, Word, and Interference conditions

	Condition	Mean (SD)
Time	Dot	14.62 (3.97)
	Word	20.15 (5.58)
	Interference	35.32 (12.93)
Errors	Dot	0.21 (0.70)
	Word	0.34 (0.77)
	Interference	1.98 (2.27)

Notes: These data should not be used as norms because they do not take into account the contribution of age, education, and sex to task performance. The reader is referred to Tables 2–6 for the normative data of this study.

Regression analyses showed that Age (β = 0.209, p < .001) and Sex (β = 0.206, p < .001) explained a significant amount of variance of the completion time of the “Dot” condition, R² = .097, F(3, 642) = 22.963, p < .001. Furthermore, analyses showed that Age (β = 0.310, p < .001), Sex (β = 0.213, p < .001), and Education (β = −0.190, p < .001) explained a significant amount of variance of the completion time of the “Word” condition, R² = .200, F(3, 640) = 53.396, p < .001. Finally, results indicated that Age (β = 0.334, p < .001), Sex (β = 0.144, p < .001), and Education (β = −0.145, p < .001) explained a significant amount of variance of the completion time of the “Interference” condition, R² = .175, F(3, 641) = 45.197, p < .001. For the “Word” and “Interference” conditions, more educated people performed faster than less educated ones. For all conditions, women and younger people had better performance (lower completion time).

Regression analyses also showed that Age (β = 0.151, p < .001) and Education (β = −0.197, p < .001) explained a significant amount of variance of Interference index score “Word/Dot for time”, R² = 0.073, F(3, 640) = 16.845, p < .001. Results finally revealed that Age (β = 0.218, p < .001) and Education (β = −0.136, p = .001) explained a significant amount of variance of Interference index score “Interference/Dot for time”, R² = .078, F(3, 641) = 18.101, p < .001. For both interference index scores, older and less educated people showed higher interference effects.

Equations to calculate Z scores for completion time on all condition and interference index scores are presented in Tables 2 and 3.

Table 2.

Equations to calculate Z scores for completion time on “Dot,” “Word,” and “Interference” conditions

Condition	Equations to calculate Z scores
	Z = (real score − expected score)/square root of the mean square residual
	Expected score	Square root of the mean square residual
Dot	0.117A + 1.660S − 0.039E + 5.705	3.784
Word	0.244 A + 2.414 S - 0.256 E + 4.345	4.998
Interference	0.609A + 3.795S − 0.455E− 5.693	11.776

Notes: One should note that a positive Z score for time means poorer performance than what is normally expected.

A = Age (years), S = Sex; F = 0: M = 1, E = Education (years).

Table 3.

Equations to calculate Z scores for interference index scores

	Equations to calculate Z scores
	Z = (real score − expected score)/square root of the mean square residual
	Expected score	Square root of the mean square residual
Word/Dot	0.007A − 0.000S − 0.015E + 1.110	0.302
Interference/Dot	0.025A − 0.033S− 0.027E + 0.986	0.777

Notes:One should note that a negative Z score here means better performance score than what is normally expected.

A = Age (years), S = Sex; F = 0: M = 1, E = Education (years).

Age was significantly correlated with error scores on “Dot” (r = .08, p = .038), “Word” (r = .15, p < .001), and “Interference” (r = .26, p < .001) conditions. Also, education was significantly correlated with error scores on “Interference” condition (r = −0.21, p < .001). Finally, sex was not significantly correlated with any condition. Percentiles for error scores are provided in Tables 4–6 for “Dot,” “Word,” and “Interference” conditions, respectively.

Table 4.

Percentiles for error score on Dot condition

Age	n	Percentiles
		1	2	5	10	15	25	50	95
≤65	72	1	1	0	0	0	0	0	0
66–72	102	5	2	1	0	0	0	0	0
73–79	301	4	2	1	1	1	0	0	0
>80	171	3	2	2	1	1	0	0	0

Table 6.

Percentiles for error scores on Interference condition

Age	Education	n	Percentiles
			1	2	5	10	15	25	50	95
≤65	≤12	20	4	4	4	2	2	2	0	0
	>12	52	6	6	3	3	2	1	0	0
66–72	≤12	36	6	6	6	4	3	2	1	0
	>12	66	4	4	4	3	2	2	0	0
73–79	≤12	158	12	9	7	5	5	4	2	0
	>12	143	10	7	5	4	3	3	1	0
>80	≤12	104	14	13	8	7	6	4	3	0
	>12	66	10	10	8	6	5	4	2	0

Finally, in order to test the potential effect of recruiting sites on performance, a variable “site” was added to the regression model for each of the dependent variable. These further analyses revealed that recruiting sites had no significant impact on the completion time of the three conditions or on the two interference index scores (p-values ranging between .179 and .956).

Discussion

The aim of the present study was to assess the effect of sociodemographic variables on performance on the French adaptation of the VST and to establish normative data for French-Quebec adults and elderly people. The normalized scores included the completion time and error scores on “Dot,” “Word,” and “Interference” conditions, as well as low (Word/Dot for time) and high (Interference/Dot for time) interference index scores.

As expected, age was found to be associated with completion time on all test conditions, which is generally in line with the results obtained by previous normative studies of Stroop task performance (Amato et al., 2006; Bayard et al., 2011; Ivnik et al., 1996; Moering et al., 2004; Troyer et al., 2006; Van Boxtel et al., 2001; Van der Elst et al., 2006). In our study, age was also related with error scores on all test conditions. Troyer and colleagues (2006) also found an association between age and error scores, although the association was limited to the Interference condition. Finally, age was found to be associated with interference index scores, with older people showing a higher interference effect. These results were also obtained by Troyer and colleagues (2006) and could be attributed to the general effect of cognitive slowing or to a decline in inhibitory control with normal aging (e.g., Bugg, DeLosh, Davalos, & Davis, 2007).

Education was found to be associated with completion time on “Word” and “Interference” conditions. More precisely, higher education level was associated with shorter completion time and with lower error scores in these conditions, which is party consistent with the study conducted by Bayard and colleagues (2011), who found an association between education and completion time scores. Moreover, education was related with error scores on Interference condition, with less educated people obtaining lower scores, and with interference index scores, with less educated people showing a higher interference effect. These results are generally consistent with previous normative studies for the Stroop test, which showed the influence of education level on performance (Bayard et al., 2011; Moering et al., 2004; Steinberg et al., 2005; Van Boxtel et al., 2001; Van der Elst et al., 2006).

Finally, we found that sex was associated with completion time on all conditions, with women performing faster than men on all conditions of the test. Although no normative study found sex differences in VST performance, some researchers found women to out-perform men in other versions of the Stroop test (Strickland et al., 1997; Van der Elst et al., 2006). Interestingly, a previous study suggested that women generally show higher processing speed than men (Camarata & Woodcock, 2006). However, further research is needed to clarify the effect of sex on completion time in the VST.

According to Bayard and colleagues (2011), who provided normative data in elderly French speakers from France, sex had a negligible association with completion time and error scores on all conditions. Furthermore, age and education were not significantly associated with error scores. In our sample, however, sex was significantly associated with completion time on all conditions, with women completing all conditions faster than men. Age and education were also significantly associated with error scores, with younger and more educated people performing better than older and less educated ones. Nevertheless, one should note that education was divided dichotomously in Bayard and colleagues (level 1 = <12 years; level 2 = ≥12 years), whereas it was entered as a continuous variable in our study. Besides, according to Troyer and colleagues (2006), who presented normative data for English speakers from Canada, age was the only variable associated with performance on the VST. This highlight the importance of developing normative data adjusted to the cultural and linguistic reality of Quebec-French individuals.

An original aspect of this study was the use of a regression-based approach to calculate normative data for completion time and interference scores. In fact, although these equations differ from typical normative methods, such as percentiles or standard Z scores calculated from a mean and an SD, this approach leads to better estimation of an individual's performance given his/her specific characteristics. To illustrate the use of these equations in clinical practice, let us imagine a 75-year-old man with 10 years of education, who had completed the Dot, Word, and Interference trials in 20, 24, and 45 s, and made 0, 1, and 2 errors, respectively. According to the regression equations, his expected times would be 15.75, 22.50, and 39.23. To calculate Z score, the expected score is subtracted from the real score, and then divided by the square root of the mean square residual. As such, Z scores for each trial would be 1.12, 0.30, and 0.49, which is considered as clinically normal. Here, one should note that a positive Z score means a poorer score than what is actually expected, because higher scores represent higher completion time. Next, according to regression equations, his expected low and high interference scores would be 1.49 and 2.56, and Z scores would be −0.94 and −0.40, which is again considered as clinically normal. Looking at Tables 3–5, error scores are within normal ranges, namely over the 15th percentile. In order to make it easier to use, we suggest using automatic formulas available by contacting the corresponding author of this study.

Table 5.

Percentiles for error scores on Word condition

Age	n	Percentiles
		1	2	5	10	15	25	50	95
≤65	72	1	1	1	0	0	0	0	0
66–72	102	3	2	1	1	0	0	0	0
73–79	301	4	2	2	1	1	1	0	0
>80	170	4	3	2	2	1	1	0	0

The main limitation to the present study was the use of an incidental sampling method, which resulted in an underrepresentation of people aged between 50 and 60 years, an overrepresentation of female participants and in a higher educated sample. Although a random sampling method would have been ideal, the sampling of the present study is a starting point in the establishment of VST norms for the Quebec-French population which were, until now, nonexistent. As such, results should be interpreted carefully in individuals within the youngest (≤65) age cluster, in low-educated individuals as well as in men. Also, normative data should only be used for individuals who have been tested using the same protocol (i.e., French adaptation of the VST; Moroni & Bayard, 2009). Furthermore, although all examiners used the same task instructions and versions, and were all trained to administer the VST, no common training was conducted. Consequently, some variability may exist between them. Finally, since mild cognitive impairment is not always detected by global cognitive screening measures (e.g., MMSE or MoCA), it remains possible that some participants included in this study were in early stages of dementia, which could potentially result in reduced mean or increased variability of performance.

Funding

This work was funded by the Réseau québécois de recherche sur le vieillissement, the Canadian Institutes of Health Research, and the Natural Sciences and Engineering Research Council of Canada. O.P. was supported by a Fellowship award from the Canadian Institutes of Health Research, M.P.T. was supported by a scholarship from the Centre de recherche sur le cerveau, le comportement et la neuropsychiatrie. C.H. and N.C. were supported by a salary award from the Fond de recherche du Québec-Santé.

Conflict of Interest

None declared.