The TMA-93: A New Memory Test for Alzheimer’s Disease in Illiterate and Less Educated People

Abstract

The present study aimed at validating the Memory Associative Test of the district of Seine-Saint-Denis (TMA)-93, a new test of episodic memory. The TMA-93 was proposed to mostly less educated and multicultural elderly population composed of 376 healthy controls (HC) and 94 patients with Alzheimer’s disease (AD). The construct validity was checked by studying correlations with a widely used memory test (the Free and Cued Selective Reminding Test [FCSRT]) in the subsample of literate patients. Results showed that (i) all the TMA-93 scores of the patients with AD were lower than those of the HC, (ii) the TMA-93 total score identified patients with AD with a high sensitivity (88%) and very high specificity (97%), and (iii) the TMA-93 total score was strongly correlated with both free recall and total recall scores of the FCSRT. Taken together, results showed that the TMA-93 is a reliable tool to assess episodic memory in a multicultural, less educated, or illiterate population, with good construct validity for AD diagnostic accuracy.

Introduction

Clinical diagnosis of Alzheimer’s disease (AD) is based on the establishment of cognitive decline, especially of episodic memory in the case of typical amnestic forms, sufficient to interfere with daily living activities. ^1,2 To objectify this memory decline, neuropsychological testing is required. However, in some cases in clinical routine, the usual neuropsychological assessment procedures are not possible or reliable; for example, when patients are illiterate or have a low level of education, neuropsychological evaluation represents a real challenge due to the lack of suitable tools. ^3,4 Normative data and the validation of classical cognitive tests have been studied in literate participants only and often require abilities that have been acquired at school such as reading and writing. ⁵

Neuropsychological tests to assess episodic memory usually involve learning and recalling a list of words or stories. ^{6 -8} When demographic effects on these tests are taken into account, it appears that performance is more impacted by participants’ educational level than by their age. ⁹ Moreover, the words used in these tests might not be appropriate when participants are from other cultures and/or have a native language that differs from that of the normalization/validation sample. For example, in the French adaptation of the Free and Cued Selective Reminding Test (FCSRT), ^10,11 which is widely used in memory clinics in France, ¹² the words chosen were controlled for their phonological complexity, length, and typicality rank in French (as being familiar enough but not prototypical). Added to the fact that the French FCSRT requires reading abilities, not being a native French speaker and having a cultural background that differs from the metropolitan French, one will intrinsically change the typicality rank of words and consequently the difficulty of the task (eg, a daffodil is not a familiar flower for African people). Lastly, participants with no or less education preferentially learn by heart rather than by using encoding strategies. ¹³ For example, semantic clustering is rarely implemented while encoding or retrieving information. ⁹

Taking these limitations together, it seems inappropriate to use classical memory tests for illiterate and/or multicultural patients since a lower performance might be expected in these patients, increasing the risk of false-positive errors in the diagnosis of AD. ¹⁴ Yet, illiteracy and a low educational level are now well known to be important risk factors for AD. ^{15 -17} It is thus crucial to develop new memory tests for these populations. A few previous studies suggested that procedures using the recall of drawings or images of real objects might be more suitable, especially when drawings refer to objects that are familiar to patients. ^18,19 Based on these findings, we recently validated a new episodic memory test (TNI-93 or “Test des Neuf Images du 93,” ie, Nine Images test of the district of Seine-Saint-Denis) ²⁰ for the screening of dementia in illiterate and less educated patients. Although psychometric properties of this first test were found to be good for the screening of dementia in general, a test for the screening of AD in particular was still needed. We designed a new test dedicated to this objective in illiterate and less educated people: the Memory Associative Test of the district of Seine-Saint-Denis (TMA-93) ²¹ assessing episodic memory. The TMA-93 was based on the principle of the associative learning strategy used in the subtest of verbal paired associates from the Wechsler Memory Scale. ⁸ Drawings of familiar objects of everyday life are displayed in semantically related pairs during the encoding phase. Then, participants have to recall the missing drawing when the associated drawing is provided. This retrieval phase is repeated 3 times successively to assess participants’ learning abilities.

Although demographic effects have already been studied to provide normalization data in healthy illiterate people, ²¹ a clinical validation study in pathology was needed. The present study aimed at investigating (i) whether the TMA-93 shows good properties for AD diagnostic accuracy in a population living in France but with a low level of literacy or education and (ii) the construct validity of the TMA-93 using correlations between the TMA-93 and the French adaptation of the FCSRT scores in a sample of native French speakers and literate patients.

Methods

Participants

A total of 470 participants (healthy controls [HC]: n = 376; patients: n = 94), aged older than 60 years, were included. All participants have signed informed consent, and the study was carried out following the Declaration of Helsinki principles.

The whole sample was multicultural, composed of native French speakers with different levels of education and of participants with an immigrant background, mainly less educated with variable levels of French language skills, especially in its written form. The HC were included over a 32-month period at the Centre d’Examen de Santé (CES; Center of Health Exams) in Bobigny. All of them lived in the Seine-Saint-Denis district (France). They were recruited on a voluntary basis during a free health checkup proposed by the French National Health Service. During the checkup, they were examined by a neurologist who excluded a diagnosis of dementia or psychiatric illness based on the clinical criteria of the Diagnostic and Statistical. Manual of Mental Disorders (Fourth Edition). ¹ Participants with severe visual and/or auditory disorders that could interfere with neuropsychological testing were excluded, as were participants with a history of neurological or psychiatric disease. When memory disorders were suspected, the participant was systematically excluded and addressed to the Memory Clinic for further investigations.

Patients attending the Memory Clinic at the Department of Neurology of Avicenne Hospital (Assistance Publique des Hôpitaux de Paris) over the same period and who met the clinical diagnosis of AD according to the DSM-IV criteria ¹ were included. The diagnosis of AD was made by a neurologist, who was blind to the patient’s TMA-93 results. When the patient’s level of French permitted (85% of the patients), the French version of the Mini-Mental State Examination (MMSE) ^22,23 was proposed. Demographic characteristics are summarized in Table 1.

Table 1.

Participants’ Characteristics.^a

	Healthy Controls (n = 376)	Patients With AD (n = 94)	P Value
Age (years)	68.7 (5.9)	77.9 (6.5)	<.001
Sex (male/female)	180/196	30/64	.002
Education level			.25
Illiterates	68	19
1 to 7 years	148	41
8 to 11 years	97	15
≥12 years	63	19
MMSE (of 30)	-	19 (5.1)b

Abbreviations: AD, Alzheimer’s disease; MMSE, Mini-Mental State Examination.

^aThe values are expressed as mean (standard deviation).

^bMissing data for 14 patients.

The 2 groups differed in age (t ₄₆₈ = −13.2; P < .001; Cohen d = 1.32) and sex distribution ( ${\chi }_{1;\hspace{0.17em}\text{N}=470}^2$ = 5.8; P = .02; Cramer V = 0.13). Patients with AD were older than HC, and there was an overrepresentation of women compared with HC. However, education level was similar between groups ( ${\chi }_{1;\hspace{0.17em}\text{N}=470}^2$ = 4; P > .2; Cramer V = 0.09).

Material and Procedure

The TMA-93 comprised black and white drawings from the Snodgrass and Vanderwart database. ²⁴ Twenty drawings of familiar objects of everyday life were selected (Figure 1) and pretested on a sample of healthy volunteer controls from the CES. This sample, aged 60 years or older, shared the same sociocultural characteristics as participants included in the validation study. All drawings were easily identified and orally named in French by the whole pretested sample, even when they had no formal education and/or did not master the French language. Paired associates were then constituted such that each paired drawing was semantically related (eg, “tree” and “bird”; “knife” and “apple”; Figure 1), but all the associates refer to different semantic categories.

Figure 1.

Illustration of the TMA-93 procedure: (A) examples of paired associates to name and memorize and (B) presentation of the first stimulus during the first cued recall.

The TMA-93 is a test assessing episodic memory, with the following advantages: (i) instructions are easily understandable including for participants with a low level of French; (ii) the test is short, taking around 10 minutes to administer; and (iii) it is not necessary to master written French or even to have a high level of spoken French since responses can be made in the participant’s mother tongue. During the encoding phase, participants were asked to orally name the 10 paired drawings successively (the tree and the bird) and were explicitly told to memorize them. After this encoding phase, the first cued recall was proposed: For each pair of associates, only 1 drawing was displayed (eg, the tree) and participants had to recall the missing associated drawing (eg, the bird; Figure 1). After the participant’s response, whatever its accuracy in a 5-second delay, the previously encoded paired drawing was presented again. If no response was given, the participant was asked to name the missing drawing again. The procedure was the same for the 9 other paired associates.

When participants did not accurately recall the drawing of the 10 paired associates during the first cued recall (ie, giving a score <10), a second and a third cued recall were proposed, following the same procedure, leading to a total score of about 30 (by summing the number of correct responses on the 3 recall phases). On the contrary, when participants obtained the maximum score of 10 after the first cued recall (or the maximum score of 20 after the second cued recall), the procedure was stopped, and 20 points (or the remaining 10 points) were automatically credited for a total score of 30.

Several kinds of errors were distinguished: (i) we called “errors” all responses corresponding to an object that belonged to 1 of the 9 other paired associates, (ii) “intrusions” corresponded to responses that did not belong to the 10 paired associates, and (iii) “perseverations” referred to repeatedly produced errors during the whole procedure.

Normative data of the TMA-93 were established in a multicultural population, mainly of a low level of education among volunteers at the CES. Demographic characteristics were, therefore, comparable to those of the sample included in the present study who came from the same center. We briefly report here the main results that have been previously described elsewhere. ²¹ Normative data were obtained based on the results of 433 participants (68.5 ± 6 years), of whom 59.4% were native French speakers. Concerning participants’ education level, 15.7% had had no schooling and were illiterate, 20.6% had a primary school level (between 1 and 7 years of education), 46.5% had a secondary school level (between 8 and 11 years of education), and only 17.2% had completed secondary education, with or without university degrees. This stratification has been chosen following the French educational system (primary school, secondary school corresponding to the middle and high school until the baccalaureate, then higher education after the completion of secondary school, that is, 12 years of education) and the usual stratification found in French norms for neuropsychological tests. The study of demographic effects showed that age and sex did not impact the TMA-93 total score, but that education level did. Thus, the scores corresponding to the 5th percentile differed according to the participants’ education level, with a total score of 23 of 30 for illiterates, 24 of 30 for less educated, and 25 of 30 for secondary and university levels. ²¹ Intrusions were rare since 6.7% produced 1 intrusion or more and only 1.8% produced 2 intrusions (no healthy participant produced more than 2).

In order to investigate correlations between the TMA and the FCSRT, we selected a subgroup of patients with AD (n = 55) who underwent the FCSRT during their neuropsychological testing. This subgroup included 20 males and 35 females, aged 76.6 ± 8 years, 93% of whom were native French speakers (due to the high level of written French required for this test). Only 4 had no schooling, 28 had a primary school level, 8 a secondary, and 15 a university level. In the French version of the FCSRT, ¹¹ 16 printed words have to be learned. After a semantic cued encoding phase, followed by an immediate recall, patients were asked to free recall the 16 words. Then a cued recall (using the semantic category) was proposed for words not retrieved by free recall. This procedure was repeated 3 times, leading to a total free recall (maximum score of 48) and a total recall (total free recall added to the total cued recall; maximum score of 48).

Statistical Analyses

We first conducted between-group comparisons using Student t tests to verify that age and sex did not affect the TMA-93 performances in HC. We then compared performances of patients with AD to those obtained by HC on each TMA-93 score using univariate analysis of variances with group (HC; patients with AD) as between-group factor separately on each score (total score, errors, intrusions, and perseverations). Properties of the TMA-93 for the detection of AD were then assessed by calculating its sensitivity, specificity, and positive and negative predictive values. The Youden index was used to identify the cutoff score corresponding to the best compromise between sensitivity and specificity. Finally, Pearson correlation analyses were conducted between the TMA-93 total score and FCSRT scores.

Performances are expressed by mean ± standard error of the mean. Analyses were performed using SPSS version 18.0 (SPSS Inc, Chicago, Illinois), and the threshold for statistical significance was set at P < .05.

Results

Preliminary Analyses

Considering that our study groups differed in age and sex distribution, we first checked whether these variables impacted performances in HC. Male and female performance did not significantly differ on the total score (28.2 ± 0.14 and 28.5 ± 0.14, respectively; t ₃₇₄ = −1.5; P > .1; Cohen d = 0.16), errors produced (0.4 ± 0.06 and 0.28 ± 0.06; t ₃₇₄ = 1.5; P > .1; Cohen d = 0.15), or intrusions (0.1 ± 0.03 and 0.06 ± 0.02; t ₃₇₄ = 1.7; P > .08; Cohen d = 0.18). Controls did not produce perseverations. To control for a potential age effect, we constituted 2 groups based on the median age (60-68 years vs ≥69 years). The 2 groups did not differ on the total score (28.4 ± 0.14 and 28.4 ± 0.15, respectively; t ₃₇₄ = −0.1; P > .9; Cohen d = 0.001), errors produced (0.3 ± 0.05 and 0.38 ± 0.06; t ₃₇₄ = −1.2; P > .2; Cohen d = 0.12), or intrusions (0.08 ± 0.03 and 0.09 ± 0.02; t ₃₇₄ = −0.3; P > .7; Cohen d = 0.03).

Effect of Pathology on TMA-93 Performances

The scores obtained on the TMA-93 in both groups are summarized in Table 2. They revealed that performances of patients with AD were lower regardless of the score considered. They recalled fewer drawings than HC (F _{1, 468} = 1109; P < .001; η² = .7) and produced more errors (F _{1, 468} = 43.3; P < .001; η² = .09), intrusions (F _{1, 468} = 176; P < .001; η² = .27), and perseverations (F _{1, 468} = 123; P < .001; η² = .21) than HC did.

Table 2.

TMA-93 Performances in Healthy Controls Versus Patients With Alzheimer’s Disease (AD).^a

	Healthy Controls (n = 376)	Patients With AD (n = 94)	P Value
Total score (of 30)	28.4 ± 0.2	12.2 ± 0.4	<.001
Errors	0.34 ± 0.05	1.14 ± 0.11	<.001
Intrusions	0.09 ± 0.07	2.11 ± 0.14	<.001
Perseverations	0	0.91 ± 0.07	<.001

^aThe values are expressed as mean ± standard error of the mean.

Properties of TMA-93 for the Diagnosis of AD

Discriminative properties for patients with AD were studied using the TMA-93 total score. The Youden index was calculated to identify the best threshold value of the total score for the diagnosis of AD. It revealed that the highest value was obtained with the cutoff of 24 of 30 (Youden index of 0.84). When considering the total score <24 of 30 as a threshold value, the performance of 83 of the 94 patients was considered as abnormal (88% sensitivity) and the performance of 365 of the 376 HC was considered as normal (97% specificity). The positive predictive value of the TMA-93 total score was 88% and its negative predictive value was 97%. Overall, the TMA-93 total score showed good properties for the diagnosis of AD.

Correlation Analyses

To verify the construct validity of the TMA-93 total score, we then studied correlations between this index and performances on the widely used FCSRT in a subgroup of patients with AD (n = 55, mean age: 77.9 ± 6.5 and mean MMSE: 19.1 ± 5.2). In this subgroup, the mean TMA-93 total score was 15.4 ± 1.2, the FCSRT total free recall (of 48) was 7.8 ± 0.8, and the FCSRT total recall (of 48) was 25.4 ± 1.4.

The TMA-93 total score was positively and strongly correlated with the FCSRT total free recall (r = .63; P < .001) and with the FCSRT total recall (r = .51; P < .001). We also checked how many patients were considered as normal on the TMA-93 (based on the previously calculated threshold value) but abnormal using the French normative data of the FCSRT. ¹¹ It was found that 9 patients (ie, 16%) were in this case. When looking more closely at their characteristics, it appeared that most of them (8 of 9 patients) had a secondary or university level of education, suggesting that the TMA-93 total score might not be sensitive enough when participants had a moderate to high level of education.

Discussion

The main aim of the present study was to validate a tool ²¹ previously proposed to assess episodic memory in illiterate or less educated people and to investigate its sensitivity and specificity for the diagnosis of AD. First of all, it is noteworthy that the TMA-93 was well understood in our sample, even when participants had no schooling, had a poor level of spoken French, and/or presented cognitive disorders. Added to the fact that this is a short test with no minimum level of reading and writing, this result confirms that it is easily understandable and well adapted for such a population. Second, we showed that the TMA-93 total score has good sensitivity and specificity for the detection of AD. The present clinical validation study indicates a good sensitivity and specificity to detect AD in a multicultural population mainly composed of less educated people. Interestingly, although we did not specifically study this score in the present control sample, normative data of the TMA-93 showed that intrusions were extremely rare in HC. This suggests that the production of more than 2 intrusions might be a sign of memory disorder, sometimes considered as a pathognomonic sign of AD. ²⁵

The TMA-93 is based on associative learning and uses visual presentation. These 2 properties might account for our interesting results. It has been found that deficits in visual memory are a good predictor of AD occurrence several years before diagnosis. ²⁶ Consistently with the present findings, Lindeboom et al ²⁷ reported good properties of another test in the detection of AD in less educated people. The Visual Association Test used also consisted of associative material: Pairs of semantically unrelated objects were displayed (eg, an ape holding an umbrella) during the encoding phase, then participants were asked to recall the missing objects of each pair (only 1 object was presented as a cue for each pair).

In a validation perspective, the present study also aimed at investigating the construct validity of the TMA-93. Correlations between this test and the French adaptation of the FCSRT scores were conducted in a sample of native French speakers and literate patients (with moderate to high level of education). As expected, strong and positive correlations were found between the TMA-93 total score and both free recall and total recall scores of the FCSRT. These results argue strongly in favor of the interest of TMA-93 for the detection of AD since it has been previously found that impairment of free recall and total recall scores of the FCSRT (i) correlated with hippocampal atrophy in AD ²⁸ and (ii) can identify prodromal AD in patients with mild cognitive impairment with a high sensitivity and specificity. ¹² This suggests that the TMA-93 might be interesting not only in the detection of AD at the dementia stage (as was the case for the patients included in the present study). However, future studies are needed to further investigate its properties in the detection of early stages and to ascertain whether it could be extended to atypical presentations of AD ²⁹ or to other dementias, which are frequently associated with memory disorders. ³⁰ When we compared the TMA-93 and FCSRT scores to detect patients with AD, we showed that the FCSRT scores detected 16% of them better than the TMA-93 total score did. This might be considered as a limitation. However, when looking more closely at the characteristics of the patients who were not correctly classified by the TMA-93, it was found that 89% of them had a secondary or university level of education. Taken together, these results indicate that the TMA-93 might be an alternative to the FCSRT when neuropsychological testing concerns a less educated and/or illiterate person (the population for whom the test was designed) but cannot replace the FCSRT when the education level is higher. Correlations between the TMA-93 and the picture version of the FCSRT ³¹ might have been more appropriate to investigate the construct validity of our new tool, since both tests use drawings while the French FCSRT only use words. However, norms for the picture version of the FCSRT in a French sample are not available. Furthermore, the word version of the FCSRT is the most frequently used in France, and recent studies showed that scores from the picture and word versions are correlated. ^32,33

The main limitation of the study is that our 2 groups were not well matched on demographic variables other than educational level (ie, age and gender). We checked that these 2 variables did not impact the TMA-93 scores in the HC group. Consistently, a large epidemiologic study found that memory skills were quite well preserved in normal aging. ³⁴ Further studies are required to confirm the absence of an age effect on performances since this point would be of interest to assess older participants (ie, the “oldest-old”). This specific population is highly concerned by neuropsychological testing, given that age is a main risk factor for AD. ^35,36 Yet, diagnosis in the oldest-old is also challenging in view of several sources of vulnerability (eg, physical, sensorial, and cognitive) and the lack of normative data for people older than 85 for most neuropsychological tests. ^{37 -40}

In summary, the TMA-93 appears to have several advantages for memory assessment in a less educated and/or multicultural population. French normative data are available for this test, ²¹ and the present study strongly argues in favor of its interest in pathology given its good properties for the detection of AD. Several global scales have been published worldwide to assess such a population, especially in India, ^41,42 China, ⁴³ Mexico, ⁴⁴ Tunisia, ⁴⁵ Senegal, ⁴⁶ and Australia. ⁴⁷ However, there are few tools available to specifically assess cognitive functions that are suitable for the neuropsychological evaluation of less educated or illiterate populations. ^{48 -52} To assess episodic memory, a few tools have been adapted or specifically developed, for example, in India, ¹⁹ the Netherlands, ²⁷ and Brazil, ^18,53 but very few in French-speaking countries. ^{54 -56} To fill this gap, we also developed the TNI-93. ^20,21 The TNI-93 also assessed episodic memory. Our validation study showed its high sensitivity and specificity to screen for dementia (not specifically AD). Although the TNI-93 was validated in mostly multicultural, illiterate, and less educated people, both normative data ²¹ and the validation study ²⁰ indicated that the TNI-93 scores were not influenced by the level of education. On the contrary, normative data showed a significant effect of education level on the TMA-93 scores. ²¹ This was confirmed in the present study by the TMA-93’s lower detection power of patients with AD compared to the FCSRT scores when education level was higher. Thus, we specifically recommend using the TMA-93 to assess episodic memory in less educated people and when looking specifically for the detection of AD in this population. Finally, to complete the neuropsychological testing of illiterate and less educated populations, further studies are needed to develop and validate tools adapted to assess other cognitive functions, which are also frequently impaired and may contribute to the diagnosis of dementia, such as executive functions ^57,58 and social cognition. ^59,60