One Size Does Not Fit All: Comparative Diagnostic Accuracy of the Rowland Universal Dementia Assessment Scale and the Mini Mental State Examination in a Memory Clinic Population with Very Low Education

Abstract

Background

Diagnosing dementia in elderly immigrants is often difficult due to language and cultural barriers, low education, and illiteracy. We compared the diagnostic accuracy of the Rowland Universal Dementia Assessment Scale (RUDAS) to that of the Mini Mental State Examination (MMSE).

Methods

A total of 144 patients (42 with intact cognition, 44 with mild cognitive impairment [MCI], and 58 with dementia) were administered both instruments and were diagnosed by specialists blinded for MMSE and RUDAS results.

Results

Areas under the curve for discriminating intact cognition from MCI and dementia were comparable for RUDAS (0.81; 95% confidence interval 0.74–0.88) and MMSE (0.75; 95% confidence interval 0.69–0.85). Education and literacy were not correlated with the RUDAS but had a medium-large correlation with the MMSE (rho = 0.39).

Conclusions

The study provides additional evidence for the usefulness of the RUDAS in a highly illiterate, culturally diverse geriatric outpatient population.

Introduction

Currently, more than 46 million people worldwide are diagnosed with dementia and this number is expected to increase to 131 million in 2050 [1]. Timely diagnosis of dementia is important to unlock gateways to care, promote adequate coping, treat or delay cognitive and neuropsychiatric symptoms, and prepare for the future.

In ethnically diverse elders, the diagnosis of dementia is often delayed [2]. This is probably due to various factors. First, there may be cultural barriers such as a lack of understanding that dementia is an illness rather than a consequence of normal aging, stigmatizing or shameful views on dementia, negative experiences with and limited access to care facilities, and beliefs that caring for the patient with dementia is a family or personal responsibility [1, 3, 4].

Second, immigrants sometimes have limited proficiency in the host country's language which causes a language barrier. In the Netherlands, large ethnic minority groups are formed by non-Western immigrants from former colonies (Surinam and the Netherlands Antilles) and labor migrants in the 1970s. Especially the Turkish and Moroccan immigrants lack Dutch language proficiency and are generally very low educated and often illiterate [5].

A third factor that complicates timely diagnosis in populations where illiteracy or low educational attainment is common is the limited applicability of standard cognitive tests. However, cognitive tests are indispensable for dementia diagnosis [6]. Illiteracy profoundly affects performance on neuropsychological tests [7]. Literacy encompasses more than being able to read and write; it also influences knowledge about how and when to apply literacy skills (meta-linguistic awareness), as well as phonological skills, hand-motor sequences, declarative and working memory, executive functions, and visuospatial abilities [7, 8]. As a result, people with illiteracy often perform worse on tasks like naming, comprehension, verbal abstraction, figure recognition, and orientation [7, 8]. Furthermore, illiterate people exhibit less “test-wiseness”: they are less used to taking tests and less familiar with the formal procedures that are involved [8, 9]. There is ample evidence that low-educated or illiterate migrant populations perform worse on standard dementia screening tests such as the Mini Mental State Examination (MMSE) (Folstein et al. [33]) resulting in a high risk of false positive test results (i.e., low specificity) [10, 11, 12].

Thus, incidence and prevalence rates of dementia in illiterate and low-educated populations can be biased (i.e., overestimated) when they are solely based on standard cognitive tests. However, data from epidemiological research in which these biases are taken into account (e.g., by using a clinical mental status interview, informant interview, cross-culturally validated cognitive tests, or adjusted cutoff scores to reach a dementia diagnosis) show that illiteracy or lower education strongly predict earlier dementia diagnosis and dementia prevalence [1, 13, 14, 15, 16]. Livingston et al. [1] calculated that less education (defined as no secondary school education) is associated with a relative risk of dementia of 1.59 (95% confidence interval 1.26–2.01). These findings support the cognitive reserve hypothesis, which states that individual differences in the cognitive processes underlying task performance allow some people to cope better than others with brain pathology [17]. In the Netherlands, a recent study suggests that dementia prevalence might be higher in Turkish and Moroccan immigrants than in native Dutch people [18].

In summary, illiterate or low-educated elderly migrants have a higher risk of developing dementia, but at the same time diagnosing dementia in this group in clinical practice is harder since standard cognitive tests are not suitable. This “double whammy” is acknowledged by clinicians and researchers in several countries across Europe [19] and has led to the development of cognitive screening tests for culturally and linguistically diverse elderly. Cross-culturally validated cognitive screening tests for dementia include the Cross-Cultural Cognitive Examination (CCCE) [20], the Community Screening Instrument for Dementia (CSID) [14], the Cognitive Abilities Screening Instrument (CASI) [21], and the Cross-Cultural Dementia screening test (CCD) [22]. All these instruments take an administration time of 20–35 min. In clinical practice, brief dementia screening tests are preferred, which is likely why the MMSE is still the most popular dementia screening test used worldwide [11]. The only brief dementia screening test to our knowledge that was specifically designed for low-educated culturally and linguistically diverse elders is the Rowland Universal Dementia Assessment Scale (RUDAS), which can be administered in 10 min [23].¹ The RUDAS was developed in Australia and covers various cognitive domains: visuospatial orientation, praxis, visuoconstructional drawing, judgment, language, and memory. The RUDAS has been translated and validated in several countries including India, Thailand, Denmark, Lebanon, Iran, and Spain [26, 27, 28, 29, 30, 31]. In a meta-analysis of 11 studies with 1,236 participants in 6 countries [32], the RUDAS was judged to have good psychometric properties. Six of the included studies compared RUDAS and MMSE scores: receiver operating characteristics (ROC) curves for discriminating healthy from demented subjects were comparable for both instruments (area under the curve [AUC]: RUDAS 0.87; MMSE 0.86). Education and language were more highly correlated with RUDAS scores than with MMSE scores. The tested populations included low-educated people (mean years of education varied from 4 to 11 years), but only a low proportion of people with illiteracy (4–33%) [32]. Since many elderly people in Turkish and Moroccan minority populations in the Netherlands are illiterate or very low educated, it is important to determine the diagnostic accuracy of the RUDAS in an illiterate population. Chaaya et al. [26] determined RUDAS performance in an Arabic-speaking population of whom half were illiterate and found a good diagnostic accuracy of the RUDAS; however, the MMSE was not investigated in this study. Therefore, we compared the diagnostic accuracy of the RUDAS versus the MMSE in a large immigrant population with a high illiteracy rate. In this study, we translated and validated a Dutch version of the RUDAS. We included patients of non-Western (mainly Moroccan and Turkish) descent visiting 3 geriatric outpatient clinics in the Netherlands. The diagnostic accuracy of both the MMSE and RUDAS was determined by comparison with a dementia diagnosis by a specialist (i.e., the gold standard). We expected the RUDAS to perform at least as good as, or even better than, the MMSE and to be less sensitive to illiteracy and educational level than the MMSE.

Methods

Participants

Our population comprised patients referred to a geriatric outpatient clinic by their general practitioner for somatic and/or cognitive evaluation. The sample consisted of 152 consecutive non-Western patients recruited from February 2016 until July 2017 at a geriatric outpatient clinic in Amsterdam (Medical Centre Slotervaart) and two geriatric outpatient clinics in Rotterdam (Erasmus Medical Centre and Havenziekenhuis). All clinics were secondary or tertiary care institutions. Inclusion criteria were non-Western descent based on self-defined ethnicity and age ≥55 years. Exclusion criteria were serious vision or hearing disabilities or other conditions that could interfere with cognitive testing (e.g., paralysis, aphasia, or refusal to cooperate) and failure to complete both MMSE and RUDAS. Of the 152 patients, 12 subjects were excluded after completion of the test protocol; 1 because of a missing MMSE score, and 11 because of missing RUDAS scores.

Instruments

RUDAS

The RUDAS is a dementia screening test that consists of 6 subtests: registration and delayed recall of a 4-item verbally presented shopping list, visuospatial orientation (naming body parts), praxis (alternating hand movements), visuoconstructional drawing of a cube, judgment (asking which safety precautions one takes when crossing a street), and category fluency (animals) [23]. The maximum score is 30 points, with a higher score reflecting better performance and an advised cutoff score of < 23. Administration was done by trained spe­ cialized nurses assisted by a professional interpreter. For this study, the original English version was translated into Dutch and then back-translated to English. To facilitate administration, a written Turkish or Arabic version was also available for the professional interpreter. For the 54 patients speaking other languages (e.g., Urdu, Chinese, Hindi, or Tarifit, a Moroccan dialect that cannot be transcribed), the interpreter translated the RUDAS at the moment of administration.

MMSE

The MMSE is an 11-item dementia screening test that assesses orientation, registration, attention and calculation (serial sevens or spelling), recall, naming, repetition, comprehension (verbal and written), writing, and construction. The maximum score is 30 and the advised cutoff score is < 24, with a higher score reflecting better performance [33]. Illiterate subjects who were unable to perform certain test items (e.g., writing a sentence) were assigned zero points for that item. Administration was done by trained specialized nurses assisted by a professional interpreter.

IQCODE-sf

The Informant Questionnaire on Cognitive Decline in the Elderly-short form (IQCODE-sf) is a 16-item questionnaire which assesses cognitive decline and/or improvement over the last 10 years [34]. The IQCODE-sf is filled out by an informant, for example, a relative or other caregiver. Each item is rated on a 5-point scale from 1 (“much better”) to 5 (“much worse”). Ratings are averaged over the 16 items, resulting in a total score ranging from 1 to 5, with 3 representing “no change.” The IQCODE-sf is not affected by level of education [35, 36] or language of administration [37]. IQCODE-sf scores could only be obtained when the patient or participant was accompanied by a caregiver who was able to read and write Dutch.

CDR

The Clinical Dementia Rating scale (CDR) [38] is a global dementia staging instrument. Based on a semi-structured interview with the patient and an informant, cognitive performance is judged on 6 domains: memory, orientation, judgment and problem solving, commu nity affairs, home and hobbies, and personal care. The CDR domain scores and the total global CDR score are both rated on 5 levels: 0 (no cognitive impairment), 0.5 (very mild dementia), 1 (mild dementia), 2 (moderate dementia), or 3 (severe dementia). We also calculated the “sum of boxes score” (CDR-SOB) by adding all scores on each of the 6 domains, resulting in a score ranging from 0 to 18. The CDR has been used globally as a criterion for dementia severity in clinical trials, also in low-educated and culturally diverse populations [39]. In our study, the CDR was obtained by the geriatrician or neurologist based on the clinical interview. In case of missing CDR domain scores, the CDR total global score was based on the data in the patient file and the CDR-SOB was not calculated.

Other Cognitive Tests

Patients were also administered other cognitive tests which were used to contribute to the research diagnosis but were not separately analyzed. In the Amsterdam branch of the study, the Seven Minute Screen [40] and the CCD [22] were administered. In the Rotterdam branch, the CCD was administered, and depending on the referral question, other neuropsychological tests were added. Since the CCD is a test that relies on standard instructions given by a computer program, it could only be administered when patients spoke one of the available languages (Turkish, Moroccan Tarifit, Moroccan Arabic, Surinamese, Hindi, or Dutch). The CCD total probability score was used to describe different groups. This total score is based on the memory and executive functions subtests (Objects test part B and Sun-Moon test B) and gives the probability of having cognitive impairment due to dementia based on a logistic regression formula validated in a group of healthy volunteers versus dementia patients [22]. The range is 0 to 100; 0 representing a low probability and 100 a high probability.

Design and Procedure

The study design was cross-sectional. Patients were enrolled in the usual diagnostic workup consisting of an interview with patient and caregiver, physical examination by a geriatrician or neurologist, cognitive testing, and laboratory testing, all on the same day. All interviews and cognitive tests were translated by professional interpreters. Trained nurses and neuropsychologists administered cognitive tests. The order of administration of cognitive tests was as follows: first, MMSE (part of the usual clinical workup); second, RUDAS; third, other cognitive tests. Literacy was obtained by self-report, that is, patients who reported that they could not read and write were considered illiterate. Education level was both scored in number of years and in categories accordant with the International Standard Classification of Education 2011 of UNESCO [41].

Geriatricians or neurologists were asked to provide a research diagnosis which served as the gold standard in this study. They diagnosed patients as either having intact cognition, mild cognitive impairment (MCI), dementia, or other conditions (e.g., depression) based on all available clinical information on that day; except MMSE, RUDAS, and IQCODE-sf scores. MCI was diagnosed according to the core clinical criteria outlined by the National Institute on Aging-Alzheimer's Association (NIA-AA) workgroup [42], which are: concern regarding a change in cognition, impairment in 1 or more cognitive domains, preservation of independence in functional abilities, and no dementia. Dementia was diagnosed according to DSM-IV-TR criteria (American Psychiatric Association, 2000): multiple cognitive deficits in memory and at least 1 other cognitive domain that cause significant impairment in social or occupational functioning and represent a decline from the previous level of functioning. In summary, the research diagnosis was based on an interview with the patient and informant (with use of an interpreter), physical examination including psychiatric and neurological assessment, medication history, laboratory results, and other cognitive tests. Extended neuropsychological evaluation or brain imaging did not form part of the research diagnosis. Clinicians rated the degree to which they were certain of their research diagnosis on a scale from 1 (very uncertain) to 10 (very certain).

Statistical Analysis

Analyses were performed using SPSS statistical software (version 21; IBM Corp., 2012). Patients were divided into 4 groups based on the research diagnosis: intact cognition, MCI, dementia, and other conditions; only the first 3 groups were analyzed. Nonparametric analyses were used when variables were not normally distributed. Group differences in participant characteristics were analyzed using χ² tests for independence of categorical variables, the Kruskal-Wallis H test for continuous variables, and post hoc Mann-Whitney U tests to determine which groups differed from each other. Diagnostic accuracy was determined by calculating ROC curves for group status (intact cognition/MCI/dementia). Sensitivity, specificity, and positive and negative likelihood ratios for both RUDAS and MMSE predicting group status were calculated for different cutoff points. Youden's J (sensitivity + specificity − 1) [43] was used to select optimal sensitivity/specificity combinations. To examine the effect of par ticipants' demographic characteristics on RUDAS and MMSE scores, we carried out logistic regression analyses with relevant covariates (age, education, and sex) as predictors and group status as the outcome variable. To examine the associations between demographic variables and the test results, Spearman's rho correlation coefficients were calculated. Reliability was calculated using Cronbach's alpha measure for internal consistency. Effect sizes were calculated using Cohen's d.

Results

Participant Characteristics

In Table 1, demographic and clinical characteristics of the patients are presented. From the 152 included patients in the clinical sample, 8 patients were excluded from further analyses due to having “other conditions”; for example, chronic delirium, post-traumatic stress disorder, depression, or postanoxic encephalopathy. Countries of origin for the remaining 144 patients were Turkey (n = 59), Morocco (n = 55), China (n = 5), Surinam (n = 3), Iraq (n = 3), Afghanistan (n = 2), Cape Verde (n = 2), Egypt (n = 2), Iran (n = 2), Pakistan (n = 2), Algeria (n = 1), Armenia (n = 1), Azerbaijan (n = 1), Dominican Republic (n = 1), Eritrea (n = 1), Indonesia (n = 1), Russia (n = 1), Syria (n = 1), and Somalia (n = 1).

Table 1.

Demographic and clinical characteristics of participants (total sample: n = 144)

	Intact cognition (n = 42)	MCI (n = 44)	Dementia (n = 58)
Age (median, Q1–Q3), years	75 (68–78)	75 (68–81)	76 (71–79)
Female, %	60*	73*	48*
Country of origin, %
Turkey	41b	45	38
Morocco	41b	30	43
Other	19b	25	19
Years of education (median, Q1–Q3)	1 (0–5)	0 (0–5)	1 (0–6)
Educational level, %
No education	45	55	48
1–6 classes elementary school	43	34	29
Secondary education	12	11	14
Tertiary education	0	0	9
Illiteracy, %	43	52	43
Study center, %
MC Slotervaart Amsterdam	95	89	83
Erasmus MC Rotterdam	0	5	7
Havenziekenhuis Rotterdam	5	7	10
Diabetesa, %	60	52	57
Cardiovascular diseasea, %	81	77	64
CDR global scorec, %
No dementia (CDR 0)	43	2	0
Questionable dementia (CDR 0.5)	50	80	23
Mild dementia (CDR 1)	5	18	42
Moderate dementia (CDR 2)	2	0	30
Severe dementia (CDR 3)	0	0	5
CDR-SOB (median, Q1–Q3)	1.0 (0.0–2.0)***	2.3 (1.5–3.9)***	7.0 (4.5–10.5)***
CDR-SOB missing, %	17	27	16
IQCODE-sf score (median, Q1–Q3)	3.4 (3.0–3.8)***	3.9 (3.5–4.2)***	4.6 (4.3–4.9)***
No IQCODE-sf available, %	38	34	45
CCD total score (median, Q1–Q3)	7 (4–21)***	26 (15–73)**	96 (84–100)***
No CCD score available, %	33	30	57
RUDAS score (median, Q1–Q3)	24 (21–27)***	20 (17–24)***	17 (12–20)***
MMSE score (median, Q1–Q3)	19 (15–21)***	14 (11–21)***	11 (9–14)***

CDR, Clinical Dementia Rating scale; CDR-SOB, CDR “sum of boxes score”; IQCODE-sf, Informant Questionnaire for Cognitive Decline in the Elderly-short form; CCD total score, Cross-Cultural Dementia screening total score; RUDAS, Rowland Universal Dementia Assessment Scale; MMSE, Mini Mental State Examination.

^an = 142.

^bFigures do not add up due to rounding.

^cn = 143.

^*p < 0.05,

^**p < 0.01,

^***p < 0.001.

Group status was based on the research diagnosis. There were 42 patients judged as having no cognitive impairment, that is, intact cognition; they had somatic conditions, subjective memory complaints, or depressive complaints without cognitive impairment. There were 44 patients with MCI and 58 patients with dementia. The mean score for certainty of the research diagnosis was 7.4 (SD 1.4) on a scale from 1 (very unsure) to 10 (very sure); clinicians were significantly more certain of the “intact cognition” category (mean = 8.3; SD = 1.2) than of the MCI (mean = 7.0, SD = 1.5) or dementia category (mean = 7.1, SD = 1.3) (χ² (2, n = 136) = 23.4, p < 0.001); data from 8 individuals were missing. Cardiovascular diseases (such as hypertension, hypercholesterolemia, or myocardial infarction) were present in almost 75% of the sample and diabetes in more than half of the sample. 49% of the patients had not received any education and 46% were illiterate. When participants did not know the exact number of years of their education but did know their educational level (n = 10), they were assigned the mean years of education at their level. Participants that did not attend regular education but went to “Quranic school” or a language course were scored as having 1 year of education (n = 11).

The 3 groups did not differ in age, education, and literacy level, but there was a significant difference in gender distribution between the groups (see Table 1). Dementia severity (CDR-SOB) differed significantly between the 3 groups in the expected direction, that is, dementia patients were rated as having more impairment than MCI patients, and MCI patients as having more impairment than patients with intact cognition. The CDR total global scores were as follows: in the intact cognition group, 93% of the patients had a CDR score of 0 or 0.5 (i.e., no or questionable impairment) but, unexpectedly, 7% were rated as having mild or moderate impairment. In the MCI group, 98% were judged as having questionable or mild impairment and, contrary to expectations, 1 patient (2%) was judged as having no impairment on the CDR. In the dementia group, 77% of the patients were judged as having mild, moderate, or severe impairment, but unexpectedly 23% were judged as having only questionable impairments. Scores on the IQCODE were available for 87 participants, since 57 patients did not bring an informant or the informant present was not able to complete the questionnaire. Scores on the IQCODE-sf were highest for the dementia patients and lowest for the intact cognition sample, as expected. Total scores on the CCD were not available for all subjects due to the fact that the CCD is not available in all languages and was sometimes left out because of limited time or fatigue of the patients. The CCD scores differed significantly between the groups; the median CCD score for the cognitively intact group was 7, for the MCI group 26, and for the dementia group 96. The Pearson correlation between CDR-SOB scores and the CCD was 0.61, indicating a strong correlation. Scores on RUDAS and MMSE also differed significantly between the 3 groups in the expected direction (i.e., patients with intact cognition scored higher than patients with MCI, and patients with MCI scored higher than dementia patients).

Diagnostic Accuracy

Since we distinguished 3 groups of participants, dichotomous comparisons between cognitively intact and cognitively impaired participants could be carried out in different ways. Therefore, we calculated AUCs for 3 different comparisons, each being exemplary for differ ent clinical situations. First, we compared the patients with intact cognition to those with MCI or dementia (n = 144). Second, we directly compared those with intact cognition with those with dementia, leaving the MCI subgroup out (n = 100). Third, we compared the people with intact cognition or MCI with people with dementia (n = 144).

In Figure 1, the ROC curve for discrimination between cognitively intact participants versus MCI and dementia patients is depicted. This comparison is the most relevant for clinical practice in memory clinics where the first step is establishing whether a patient has any cognitive impairments. Outcomes of the other comparisons (intact cognition vs. demen tia; intact cognition + MCI vs. dementia) and effect sizes are listed in Table 2.

Fig. 1.

ROC curve for RUDAS and MMSE in 144 subjects

Table 2.

ROC curve data and effect sizes for RUDAS and MMSE for 3 types of comparisons

	Type of comparison
	intact cognition vs. MCI + dementia	intact cognition vs. dementia	intact cognition + MCI vs. dementia
AUC (95% CI)
RUDAS	0.81 (0.74–0.88)	0.89 (0.83–0.95)	0.82 (0.69–0.84)
MMSE	0.77 (0.69–0.85)	0.85 (0.78–0.93)	0.77 (0.75–0.87)
Effect size (Cohen's d)
RUDAS	1.27	1.75	1.28
MMSE	1.04	1.49	1.00

AUC, area under the curve; CI, confidence interval.

AUCs of the RUDAS are slightly higher than AUCs of the MMSE, but in all cases, 95% confidence intervals overlap. Formal comparison of the AUCs with the nonparametric approach described by DeLong showed no significant differences (p > 0.05) [44]. We also calculated ROC curves for people with illiteracy only; they were comparable to those of the whole group (data available on request). All effect sizes were large (≥1.00).

For the RUDAS, the cutoff score with the highest Youden's index was < 22 (or < 21) where both sensitivity and specificity were 74%; this is 1 point lower than the advised standard cutoff score [23]. For the MMSE, the standard cutoff score of < 24 had very high sensitivity (94%) but very low specificity (17%) in our population, implying a high risk of false positive diagnoses. The cutoff point with the best sensitivity/specificity combination according to Youden's index was < 14 (Table 3).

Table 3.

Sensitivity, specificity, Youden's index, and likelihood ratios for RUDAS and MMSE at different cutoff points (intact cognition vs. MCI + dementia)

	Cutoff point	Sensitivity	Specificity	Youden's index	LR+	LR–
RUDAS	16/17	0.36	0.98	0.34	15.24	0.65
	17/18	0.47	0.98	0.45	19.76	0.54
	18/19	0.51	0.93	0.44	7.14	0.53
	19/20	0.58	0.86	0.44	4.05	0.49
	20/21a	0.66	0.81	0.47	3.45	0.42
	21/22a	0.74	0.74	0.47	2.81	0.36
	22/23b	0.75	0.62	0.36	1.96	0.41
	23/24	0.80	0.55	0.35	1.78	0.36
	24/25	0.90	0.48	0.38	1.72	0.21
MMSE	12/13	0.48	0.90	0.39	5.04	0.57
	13/14a	0.60	0.90	0.50	6.28	0.44
	14/15	0.67	0.81	0.48	3.50	0.41
	15/16	0.69	0.76	0.45	2.88	0.41
	16/17	0.72	0.62	0.33	1.88	0.46
	17/18	0.75	0.57	0.33	1.76	0.43
	18/19	0.79	0.55	0.34	1.76	0.38
	19/20	0.83	0.48	0.31	1.59	0.35
	20/21	0.87	0.38	0.25	1.41	0.33
	21/22	0.91	0.26	0.17	1.24	0.34
	22/23	0.93	0.21	0.15	1.19	0.32
	23/24c	0.94	0.17	0.11	1.13	0.35
	24/25	0.95	0.14	0.09	1.11	0.34

Data based on the comparison between subjects with intact cognition versus patients with MCI + dementia. LR+, positive likelihood ratio; LR-, negative likelihood ratio; Youden's index, (sensitivity + specificity) − 1 [43].

^aBest cutoff score in this study based on Youden's index.

^bRegular cutoff score for RUDAS (Storey et al. [23]).

^cRegular cutoff score for MMSE (Folstein et al. [33]).

Effects of Demographic Characteristics on RUDAS and MMSE Scores

In the logistic regression analyses, age, years of education, sex, and RUDAS or MMSE score were predictors and group status was the outcome variable; see Tables 4 and 5. In the regression analysis of the RUDAS, education was not a significant predictor, whereas it was in the regression analysis of the MMSE.

Table 4.

Logistic regression analysis for group status (intact cognition vs. MCI + dementia): RUDAS

	B	SE B	Wald test	p	OR (95% CI)
Age	−0.008	0.030	0.075	0.785	0.992 (0.945–1.042)
Years of education	0.055	0.057	0.919	0.338	1.056 (0.962–1.160)
Sex	−0.166	0.442	0.140	0.708	0.847 (0.409–1.754)
RUDAS	−0.283	0.058	24.138	<0.001	0.753 (0.685–0.828)
Constant	7.626	3.010	6.419	0.011	2,051.412

B values are the estimated unstandardized regression coefficients. OR, odds ratio; 95% CI, 95% confidence interval of odds ratio.

Table 5.

Logistic regression analysis for group status (intact cognition vs. MCI + dementia): MMSE

	B	SE B	Wald test	P	OR (95% CI)
Age	0.005	0.028	0.031	0.861	1.005 (0.959–1.053)
Years of education	0.177	0.075	5.565	0.018	1.194 (1.055–1.350)
Sex	−0.365	0.441	0.685	0.408	0.694 (0.336–1.433)
MMSE	−0.251	0.054	21.786	<0.001	0.778 (0.712–0.850)
Constant	4.677	2.638	3.143	0.076	107.448

B values are the estimated unstandardized regression coefficients. OR, odds ratio; 95% CI, 95% confidence interval of odds ratio.

Correlation between RUDAS and MMSE

RUDAS and MMSE had a strong positive correlation, rho = 0.71, p < 0.001. However, scores on the RUDAS were generally higher than MMSE scores (see Table 1).

Correlations between Test Scores and Demographic Characteristics

RUDAS score had no significant correlation with illiteracy, years of education, or sex and only a small correlation with age (rho = −0.22, p = 0.008). MMSE had a medium-to-large correlation with illiteracy (rho = −0.34, p < 0.001) and years of education (rho = 0.39, p < 0.001) and a small-to-moderate correlation with age (rho = −0.22, p < 0.003) and sex (rho = −0.18, p < 0.03).

Effect of Illiteracy

Illiterate subjects scored worse than literate subjects on both RUDAS (mean 1.5 points lower) and MMSE (mean 5 points lower). The scores on RUDAS and MMSE are displayed in Figures 2 and 3 separately for literate (grey boxes) and illiterate patients (white boxes). The differences in score dispersion between literate and illiterate people were smaller on the RUDAS than on the MMSE. A large proportion of the individuals with intact cognition scored below the standard cutoff score of < 24 on the MMSE.

Fig. 2.

Boxplot of RUDAS scores for patients with intact cognition versus patients with MCI/dementia stratified by literate and illiterate subjects. Boxes represent the 25th-75th percentile, bars represent minimal and maximum values. The upper dashed line represents the standard cutoff score of < 23 and the lower dashed line the optimal cutoff score based on Youden's index in our study (< 22).

Fig. 3.

Boxplot of MMSE scores for patients with intact cognition versus patients with MCI/dementia stratified by literate and illiterate subjects. Boxes represent the 25th-75th percentile, bars represent minimal and maximum values, circles represent outliers. The upper dashed line represents the standard cutoff score of < 24 and the lower dashed line the optimal cutoff score based on Youden's index in our study (< 14).

Internal Consistency

In this study, Cronbach's α of the Dutch version of the RUDAS was 0.70, which is consid ered acceptable. Previously reported internal consistency of the RUDAS varied between 0.74 and 0.80 [31, 45].

Discussion

This study evaluated the RUDAS in a very-low-educated population of non-Western migrants (mostly of Turkish and Moroccan descent) in the Netherlands. We were able to include a high percentage of illiterate patients (46%) which is substantially more than in most earlier studies on the RUDAS. We evaluated the diagnostic accuracy for patients who were considered to have intact cognition after a first geriatric consultation, MCI patients, and dementia patients. We hypothesized that the RUDAS would outperform the MMSE in terms of diagnostic accuracy and have a weaker association with education level and illiteracy.

Our expectations were partly confirmed. AUC for RUDAS was 0.81 (95% CI 0.74–0.88) for discrimination between individuals with intact cognition and those with MCI or dementia. When the MCI group was excluded, AUC of RUDAS was 0.89 (95% CI 0.83–0.95). AUCs for the MMSE were 0.77 (0.69–0.85) and 0.85 (0.78–0.93) with and without MCI, respectively. Confidence intervals of the AUCs for RUDAS and MMSE overlapped in all cases, providing no final conclusion on which test was better in discriminating groups.

The AUCs we found for discrimination of cognitively intact versus patients with dementia (RUDAS 0.89; MMSE 0.85) was comparable to those in earlier published research. In their meta-analysis of diagnostic accuracy of the RUDAS and the MMSE, Naqvi et al. [32] reported pooled AUCs of 0.87 (RUDAS) versus 0.86 (MMSE). In two recent Spanish studies, AUCs of 0.89–0.90 (RUDAS) and 0.87–0.89 (MMSE) were found for discrimination between healthy and demented participants [29, 46]. A recent study in Lebanon reported an AUC of 0.92 for the RUDAS discrimination between healthy participants and patients with dementia [47].

As in earlier studies [48], discriminating between patients with intact cognition and MCI and patients with dementia was more difficult than discriminating between intact cognition and dementia alone (i.e., AUC is lower). In general, sharp demarcations between normal cognition and MCI, and between MCI and dementia, are difficult because they represent points at a continuum [42]. In the end, this distinction can only be made by clinical judgment weighing all relevant criteria. An important source of information on cognitive functioning for the clinician is the interview with the patient and informant. In our study, administration of the CDR formed part of this interview. Unexpectedly, clinicians judged 7% of the intact cognition patients as having mild or moderate impairment and 23% of the dementia group as having only questionable impairment on the CDR. We think this might be caused by cultural and educational factors. Research in Asian populations with the CDR shows some evidence for cultural influence on clinicians' judgment; in particular, the domains “judgment and problem solving,” “community,” and “home and hobbies,” may be harder to determine due to different cultural and social definitions of normal functioning in old age [49]. In our experience, informal migrant caregivers (especially children) are inclined to take over tasks of their elderly parents, such as financial and social matters, regardless of cognitive impairment being present. Clinicians might inadvertently judge such elderly as falsely impaired in “problem solving” due to cognitive impairment. Moreover, education may not only influence cognitive testing but perhaps also a semi-structured interview such as the CDR, for example, because wording is too difficult [50] or the meaning too hard to translate (e.g., proverbs).

Notwithstanding the importance of a thorough clinical diagnosis, a dementia screening test such as the RUDAS should make a quick distinction between the patients that are cognitively healthy and those at risk for dementia. The use of an appropriate cutoff score is important. Different cutoff scores can be chosen depending on whether “ruling in” or “ruling out” dementia is considered more important. In our study, the best cutoff point for the RUDAS based on both optimal sensitivity and specificity was < 22, 1 point lower than the standard cutoff score [23]. The diagnostic accuracy of the MMSE with the standard cutoff score of < 24 had a very high sensitivity, but an unacceptably low specificity of 17%, meaning a false positive rate of 83%. The cutoff point with both optimal sensitivity and specificity was as low as < 14 points, with a sensitivity of 60% and a specificity of 90%. Earlier studies also found that the cutoff point for the MMSE in an illiterate population needs to be lowered drastically to avoid false positives, similarly resulting in lower sensitivity [29].

In our study, the RUDAS was not associated with education and illiteracy, in contrast to the MMSE, where moderate-to-high correlations with education and illiteracy were found. This different pattern of results observed between the RUDAS and the MMSE is in line with previous research [29, 45].

Despite the inappropriateness of several items for illiterate subjects (such as reading a sentence, writing, copying a cube, serial sevens), the MMSE seems to be nearly as good a predictor of group status as the RUDAS in this population, based on AUC. Exploratory analyses of separate items of the MMSE showed that especially orientation to time (day of the week and season) and word recall were strong predictors of group status (intact cognition vs. MCI/dementia data available on request). Evaluation of the RUDAS subscales also showed that group status was best predicted by the delayed recall items, followed by verbal fluency and praxis (data available on request). Cognitive screening tests measuring delayed free and cued recall are most sensitive to cognitive impairment caused by dementia [51]. Impairment in orientation to time is also a strong predictor of cognitive decline (in elderly with an average of 10 years of education) [52] which may account for the relatively strong predictive qualities of the MMSE despite loss of items when testing people with illiteracy.

Strengths of this study include the cross-sectional design with comparisons between clinically relevant groups. The research diagnosis served as a gold standard in this study. Clinicians were asked to base their research diagnosis on information available on the first day of the evaluation at the memory clinic, thus based on anamnestic and heteroanamnestic information from the interview with an interpreter, physical examination, including neurological and psychiatric assessment, medication history, laboratory results, and cognitive screening tests. Clinicians were blinded for results of RUDAS, MMSE, and IQCODE. This procedure was followed to prevent incorporation bias and possible overestimation of diagnostic accuracy of RUDAS and MMSE. A disadvantage of this procedure is, however, that the research diagnosis might be less accurate than it would be when all clinical information including brain imaging and more extensive neuropsychological test results would have been available. On the other hand, in most general hospital-based memory clinics, assessment of biomarkers and imaging are not standard procedures, in contrast with procedures followed in academic hospitals. The research diagnosis made in this study is, although not perfect, conform usual consensus criteria. Another important strength was the inclusion of a large “hard-to-reach” sample in scientific research: illiterate immigrants [53]. The use of formal interpreters minimized the language barrier. However, assessing cognitive status via an interpreter can lead to other problems, for example extended assessment time or comprehension problems arising from the fact that not all test items are easily translatable (for example the repetition sentence in the MMSE).

Future studies might include native Dutch patients to investigate the effect of migrant status and language of administration on (cutoff) scores for both cognitive screening tests. Other improvements would be to counterbalance administration of MMSE and RUDAS in stead of administering them in fixed order as we did (i.e., first, MMSE; second, RUDAS). More subjects did not finish RUDAS than MMSE (n = 10), which is probably due to the RUDAS being the second test, when subjects were more likely to be tired or less cooperative. We did not time the duration of both tests nor did we formally assess clinician preference or patient preference for one of the two instruments. Knowing clinician and patient preference would be helpful in choosing between the RUDAS or MMSE in future studies.

Results from this study translate well to clinical practice. Based on our findings, we would advise the RUDAS as the instrument of choice in specific immigrant people that are low educated or illiterate, because while its discriminative ability is similar to the MMSE, it does not correlate with educational level and illiteracy. Also, the cutoff score with an optimal sensitivity/specificity combination for the RUDAS (< 22) seems more useful in clinical practice than the optimal cutoff score for the MMSE (< 14).

To summarize, our study provides evidence for the usefulness of the RUDAS as a cognitive screening test for cognitive impairment and dementia in a minority population where nearly half of participants were illiterate. We argue that one size does not fit all; that is, using a standard dementia screening test such as the MMSE in clinical settings does not suit all populations. The RUDAS is a dementia screening tool that is tailor-made for culturally and linguistically diverse elderly with a low educational level. The Dutch version of the RUDAS seems easily applicable and is freely available by sending a request to the authors.

Statement of Ethics

The study was approved by the institutional Medical Ethical Committees.

Disclosure Statement

The authors have no conflicts of interest.

Funding Sources

The Rotterdam branch of this study was funded by Stichting Coolsingel (grant No. 425) and by ZonMw Memorabel project 733050834.