Cognitive assessments in multicultural populations using the Rowland Universal Dementia Assessment Scale: a systematic review and meta-analysis

Abstract

Background:

Canada has a growing multinational immigrant population. Many commonly used cognitive assessment tools have known cultural biases and are difficult to use in ethnically diverse patient populations. The Rowland Universal Dementia Assessment Scale (RUDAS) is a cognitive assessment tool that was created for culturally and linguistically diverse populations. We performed a systematic review and meta-analysis to assess the psychometric characteristics of the RUDAS and to compare it with other available tools.

Methods:

We identified studies that assessed the psychometric properties of the RUDAS compared with a reference standard for diagnosing dementia or compared the RUDAS to other cognitive assessment tools. Two independent reviewers screened the abstracts and full-text articles and reviewed bibliographies for further references. We extracted data using standardized forms and assessed studies for risk of bias.

Results:

Our search resulted in 148 articles, from which 11 were included. The RUDAS was assessed in 1236 participants and was found to have a pooled sensitivity of 77.2% (95% confidence interval [CI] 67.4–84.5) and a pooled specificity of 85.9% (95% CI 74.8–92.6) yielding a positive likelihood ratio of 5.5 (95% CI 2.9–10.7) and a negative likelihood ratio of 0.27 (95% CI 0.17–0.40). A pooled estimate of the correlation between the RUDAS and the Mini-Mental State Examination (MMSE) was 0.77 (95% CI 0.72–0.81). Results of the RUDAS were less affected by language and education level than the MMSE.

Interpretation:

The RUDAS is a brief and freely available cognitive assessment tool with reasonable psychometric characteristics that may be particularly useful in patients with diverse backgrounds.

The prevalence of dementia is rising in Canada and around the world. More than 35 million people have dementia globally, and this number is expected to triple by 2050.¹ Currently, 58% of the burden of dementia worldwide is in low- and middle-income countries, with an expected increase to more than 70% by 2050.¹ Canada is a diverse country with a large immigrant population, and in 2012, more than 11 000 immigrants coming to Canada were older than 65 years.² The top 4 countries from which Canadian immigrants originate (China, the Philippines, India and Pakistan) are low- and middle-income countries.³ With the aging immigrant population in Canada and the increasing prevalence of dementia worldwide, the number of Canadian immigrants with dementia is expected to rise substantially in the coming decades. Consequently, we need to ensure that adequate cognitive assessment tools are available to maximize chances of early diagnosis and management of dementia, as this can improve clinical outcomes and quality of life, and save health care costs.^4,5

Several formal brief tools for cognitive assessment are commonly used in Canada. These include the Mini-Mental State Examination (MMSE), which was developed in 1975,⁶ and the Montreal Cognitive Assessment (MoCA), developed in 2003.⁷ Both tools are widely used, and the MMSE is one of the most commonly used cognitive instruments worldwide.⁸ Despite their widespread use, these tools have known difficulties in patients with a low level of education and in those who do not speak English as a first language.^8,9 A further limitation to using the MMSE is its current copyright status, which requires the test to be purchased for each administration.

The Rowland Universal Dementia Assessment Scale (RUDAS) was developed in 2004 in Australia and was designed specifically to address the challenges of detecting cognitive impairment in culturally and linguistically diverse populations.¹⁰ The RUDAS is a 6-item questionnaire that assesses multiple cognitive domains and can be administered in less than 10 minutes (Table 1). The original validation study of the RUDAS showed excellent psychometric properties in a multiethnic Australian population.¹⁰ This initial cross-sectional validation study included 90 participants (45 with dementia and 45 without) from the community. The mean age was 79.5 years, and less than half of the participants had more than 6 years of education. Both the interrater (intraclass correlation coefficient [ICC] 0.99) and test–retest (ICC 0.98) reliabilities of the RUDAS were very high. Compared with the criteria of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV), the RUDAS was found to have a sensitivity of 89% (95% confidence interval [CI] 76–96) and specificity of 98% (95% CI 88–97). Performance on the RUDAS was not affected by years of education (p = 0.20) or preferred language (p = 0.33).¹⁰

Table 1:

The Rowland Universal Dementia Assessment Scale

Cognitive domain	Question*	Points
Registration	Given 4 grocery items to register (and recall later)	0
Visuospatial orientation	Left/right orientation with body parts	5
Praxis	Alternating hand movements with fist and palm	2
Visuoconstructional drawing	Copying image of a cube	3
Judgment	Safety precautions when crossing a street	4
Memory recall	Recalling 4 grocery items from above	8
Language	Animal naming in 1 minute	8
	Total score	/30

^*Questions have been abridged. The full test and details are available at https://fightdementia.org.au/sites/default/files/20110311_2011RUDASAdminScoringGuide.pdf.

Given the above-mentioned limitations of the MMSE and MoCA, the RUDAS may be a cognitive assessment tool that can assist clinicians in better assessing culturally and linguistically diverse populations. In this systematic review and meta-analysis, we sought to determine whether the initially reported strong psychometric properties of the RUDAS have been reproduced in other settings (objective 1), and to determine how the RUDAS compares to other commonly used brief cognitive assessment tools (objective 2).

Methods

Search strategy

We used a broad search strategy to identify any published study that assessed the RUDAS. We included studies with adult populations (≥ 18 yr) assessed with the RUDAS, irrespective of study design, as long as original data were used.

Studies published in any language assessing the psychometric properties of the RUDAS compared with a reference standard (for the diagnosis of dementia) were included for objective 1. The reference standard could include detailed neuropsychologic testing, expert assessment or appropriate DSM criteria. For objective 2, we included any studies comparing the RUDAS with an alternative cognitive assessment tool. Studies were excluded if the RUDAS was part of the reference standard for diagnosing dementia (objective 1 only) or if they did not use the RUDAS in its entirety.

We searched multiple databases for relevant search terms (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140802/-/DC1). Reference lists of identified articles were searched manually for further relevant articles. We contacted the corresponding authors of all studies to determine whether any further articles were available and to request any missing information from their studies.

Data collection and analysis

Two independent reviewers (R.M.N. and S.H.) inspected the abstract for each study and applied the inclusion and exclusion criteria. Studies meeting the criteria then had a full-text review by the same 2 independent reviewers. Disagreements were resolved by a third reviewer (S.A.).

We extracted the following details from each article: the numbers of participants with and without dementia (based on the reference standard), the sensitivity and specificity for the RUDAS and MMSE, and the correlations between the MMSE and RUDAS. We used the method of Reitsma and colleagues¹¹ as implemented in the R-package mada¹² to obtain pooled estimates of sensitivity, specificity, positive and negative likelihood ratios,¹³ and the area under the summary receiver operator characteristic curve (AUC), and to carry out meta-regression. We used the I² statistic to measure heterogeneity.¹⁴ We obtained a pooled correlation coefficient and its 95% CI by back-transforming the random-effects summary of the Fisher-transformed correlation coefficients.

Quality assessment

Two reviewers (R.M.N. and S.H.) independently assessed the methodologic quality of the included studies using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies–2) tool.¹⁵ The tool was designed for use in systematic reviews to evaluate the risk of bias and applicability of diagnostic accuracy studies.¹⁵ Disagreements were resolved by a third reviewer (S.A.).

Subgroup analyses

A meta-regression compared studies that investigated patients within cultures with those that studied patients across cultures.

Results

Search results

The initial search yielded 148 studies. After abstract and full-text review, 11 published articles and 3 published abstracts were selected (Figure 1). Data were extracted for the full-text articles only, because insufficient data were available from the abstracts^16–18 and further data were unobtainable. Available data from the abstracts can be found in Appendix 2 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140802/-/DC1).

Figure 1:

Selection of studies for the systematic review and meta-analysis. Note: RUDAS = Rowland Universal Dementia Assessment Scale.

The 11 identified studies were completed in 6 different countries. All studies used formal interpreters when needed. One study assessed an inpatient population,¹⁹ another investigated residents of a nursing home,²⁰ and the rest assessed use of the RUDAS in older adults living in the community. A summary of data on the studies and participants can be found in Table 2.^10,19–28 A total of 1236 participants were included, with study size ranging from 46 to 204 participants. Ten studies were cross-sectional and one was case–control.²¹ Eight studies assessed the sensitivity and specificity of the RUDAS compared with a reference standard. Nine studies compared the RUDAS to another cognitive assessment tool, which was the MMSE in all studies. One study²³ also assessed the General Practitioner Assessment of Cognition (GPCOG) in addition to the MMSE and RUDAS. Because the GPCOG is a participant- and informant-based questionnaire and only assessed in this single study, no further comparison has been made in this review.

Table 2:

Summary of included articles

Study; country	Language or origin of participants	n	Age, mean, yr	Sex, % male	Education, mean (range), yr	Literacy, %	MCI, %	Dementia, % (type)	MMSE score, mean	RUDAS sensitivity and specificity (95% CI)	RUDAS–MMSE correlation (95% CI)
Studies assessing only diagnostic accuracy of the RUDAS (objective 1)
Storey et al.;10 Australia	English 33% Other 67%	90	79.5	22.5	NA	NA	0	50 (not reported)	10D 25N	Sensitivity: 89 (76–96) Specificity: 98 (88–97)	NA
Iype et al.;21 India	India (Malayalam)	116	65.1	NA	5.5	NA	0	50.0 (Alzheimer 40%, vascular 29%, mixed 5%, frontotemporal 7%, with Lewy bodies 3%, other 16%)	13.7D 22.7N	Sensitivity: 88 (77–94) Specificity: 76 (64–85)	NA
Studies assessing diagnostic accuracy of the RUDAS and comparing it with another cognitive assessment tool* (objectives 1 and 2)
Rowland et al.;22 Australia	English 34% Asia 31% Europe 25% Middle East 6% Africa 2% S. America 2%	111	79.0	28	11	79	Excluded (14%)	56.8 (not reported)	10D 25N	Sensitivity: 81 (69–90) Specificity: 96 (86–99)	0.85 (0.79–0.89)†
Basic et al.;23 Australia	English 57% Europe 38% Middle East 2% Africa 2% Asia 0.5% S. America 0.5%	151	77.1	30.5	8.7	96	22	38.4 (Alzheimer 48%, vascular 17%, mixed 19%, other 16%)	23	Sensitivity: 88 (76–95) Specificity: 90 (80–96)	0.78 (0.71–0.84)†
Gonçalves et al.;24 Australia	English 90% Other 10%	204	76.9	44	NA	NA	13	49.0 (Alzheimer 65%, vascular 8%, mixed 17%, frontotemporal 3%, with Lewy bodies 3%, other 4%)	22.3	Sensitivity: 66 (58–74) Specificity: 90 (78–96)	0.73 (0.66–0.79)†
Limpawattana et al.;25 Thailand	Thailand	178	71.0	50	NA	96	Excluded (11%)	44.5	19.5D 26.5N	Sensitivity: 62 (51–71) Specificity: 84 (75–90)	0.80 (0.74–0.85)‡
Nielsen et al.;26 Denmark	Denmark 75% Middle East 10% Asia 5% Europe 5% America 3% Africa 2%	137	69.4	52.5	9.8 (0–20)	NA	10	52.6 (Alzheimer 57%, vascular 12.5%, mixed 9.7%, with Lewy bodies 6.9%, frontotemporal 2.8%, other 11.1%)	22D 27N	Sensitivity: 64 (52–75) Specificity: 83 (71–91)	0.73 (0.64–0.80)†
Juwita et al.;27 Malaysia	Malaysia	49	68.0	44.9	NA	NA	NA	20.4 (Alzheimer 40%, vascular 60%)	28.2	Sensitivity: 80 (49–94) Specificity: 51 (36–66)	0.63 (0.42–0.77)‡
Studies only comparing the RUDAS with another cognitive assessment tool* (objective 2)
Pang et al.;19 Australia	English 61% China 17% Europe 22%	46	81.7	46	8.4	NA	0	20.0 (not reported)	20.6	NA	0.83 (0.71–0.90)‡
Nielsen et al.;28 Denmark	Turkey	76	61.6	43	3.9 (0–16)	67	0	0	23.7	NA	0.40 (0.20–0.58)†
Salari et al.;20 Iran	Iran	78	78.8	17.9	NA	7.7	0	100	NA	NA	0.63 (0.47–0.75)†

Note: CI = confidence interval, D = dementia, MCI = mild cognitive impairment, mixed = Alzheimer/vascular mixed dementia, MMSE = Mini-Mental State Examination, N = no dementia, NA = not available, RUDAS = Rowland Universal Dementia Assessment Scale, S. America = South America.

^*The alternative cognitive assessment tool in all studies was the MMSE (see “Search results” in text).

^†Spearman correlation.

^‡Pearson correlation.

Quality assessment

The overall quality of the studies was very good as assessed by QUADAS-2 (Table 3). However, there were a few limitations in the included studies. Four studies did not report clearly on the method of participant selection.^20,21,23,25 Of the 8 studies that were used to assess the psychometric properties of the RUDAS, one was a case–control study²¹ and one did not prespecify the cut-off point to determine a positive test on the RUDAS.²⁴ Two studies also excluded participants with mild cognitive impairment in their analyses.^22,25

Table 3:

Quality assessment of articles using QUADAS-2¹⁵

Study	Patient				Index test			Reference standard				Flow and timing
	Random sample	Not case–control	Proper exclusion	Patients meet criteria	Interpret without reference results	Prespecify threshold	Meet review question	Correctly classify	Interpret without index test	Meet review question	Proper interval	Reference standard for all	Same reference standard	All participants analyzed
Studies assessing only diagnostic accuracy of the RUDAS (objective 1)
Storey et al.10	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Iype et al.21	Unclear	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Studies assessing diagnostic accuracy of the RUDAS and comparing it with another cognitive assessment tool (objectives 1 and 2)
Rowland et al.22	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Basic et al.23	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Goncalves et al.24	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Limpawattana et al.25	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Nielsen et al.26	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Juwita et al.27	Unclear	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	Unclear	Yes	Yes	Yes	Yes	Yes
Studies only comparing the RUDAS with another cognitive assessment tool (objective 2)
Pang et al.19	Yes	Yes	Yes	Yes	No	Unclear	Yes	NA	NA	Yes	NA	NA	NA	Yes
Nielsen et al.28	Yes	Yes	Yes	Yes	NA	No	Yes	NA	NA	Yes	NA	NA	NA	Yes
Salari et al.20	Unclear	Yes	Yes	Yes	No	Unclear	Yes	NA	NA	Yes	NA	NA	NA	Yes

Note: NA = not applicable, QUADAS-2 - Quality Assessment of Diagnostic Accuracy Studies–2.

In the 3 studies that only compared the RUDAS with the MMSE,^19,20,28 assessors were aware of the dementia status of participants. The threshold or cut-off score of either test was also not prespecified in these 3 studies.

Demographics

The mean age of participants was 73.5 years, and 60.9% were female. The mean educational attainment was 7.3 years (reported in 6 studies, n = 637 participants). One study included only participants without dementia²⁸ and another included only participants with dementia.²⁰ Excluding these 2 studies, the proportion of participants with dementia ranged from 20% to 75%. Alzheimer dementia was the most common type (Table 2).

Objective 1: psychometric properties of the RUDAS

Eight studies assessed the sensitivity and specificity of the RUDAS and all used the DSM-IV criteria as the reference standard. All but one²⁴ used the original cut-off score (< 23). The pooled estimates were 77.2% (95% CI 67.4–84.5) for sensitivity and 85.9% (95% CI 74.8–92.6) for specificity (Figure 2). There was substantial heterogeneity among studies for both the sensitivity (I² = 66%, p < 0.001) and specificity (I² = 78%, p < 0.001). The pooled positive likelihood ratio was 5.5 (95% CI 2.9–10.7), and the pooled negative likelihood ratio was 0.27 (95% CI 0.17–0.40). The AUC for the RUDAS was 0.87 (Figure 3).

Figure 2:

Sensitivity and false-positive rate of individual studies and pooled estimate for the Rowland Universal Dementia Assessment Scale.

Figure 3:

Sensitivity and false-positive rate of individual studies and combined results for the Rowland Universal Dementia Assessment Scale (RUDAS) and Mini-Mental State Examination (MMSE), with 95% confidence regions.

In meta-regression, specificity (p = 0.01) but not sensitivity (p = 0.8) was significantly higher in the cross-cultural group. Estimated values of specificity within and across cultures were 72.5% and 91.4%; the corresponding values for sensitivity were 76.1% and 78.3%.

There was no significant difference between the 2 groups in the mean value of the correlation between the MMSE and the RUDAS (p = 0.6).

Objective 2: the RUDAS compared with the MMSE

In the 6 studies that also reported on diagnostic performance of the MMSE, the pooled estimates were 81.8% (95% CI 72.6–88.5) for sensitivity and 76.4% (95% CI 63.7–83.8) for specificity; the AUC was 0.86 (Figure 3). Meta-analysis of the 9 studies that reported a correlation between the RUDAS and the MMSE found a pooled estimate of 0.77 (95% CI 0.72–0.81) and significant heterogeneity (I² = 63.1%).

Effect of education

Four studies investigated the effect of patients’ education level on the results of the RUDAS. The initial validation study¹⁰ found no effect of education level on RUDAS performance (p = 0.20) in 90 participants, based on a multivariable regression model including age, sex, education and preferred language. One Australian study (n = 111, mean education 6.2 yr) reported a significant effect of education on the performance of the MMSE (p = 0.016) but not of the RUDAS (p > 0.05, exact value not reported).²² In a study from India (n = 116, mean education 5.5 yr), RUDAS scores had a smaller correlation (0.45) with years of education than MMSE scores (0.64).²¹ Another study, from Denmark (n = 76, mean education 3.9 yr), found that the correlation with years of education was lower for RUDAS scores (0.42) than for MMSE scores (0.76).²⁸

Effect of language

Three Australian studies investigated effects of language on RUDAS performance. The original validation study (n = 90) found no significant effect of language on RUDAS performance (p = 0.33)¹⁰ using the multivariable model described above. Another study (n = 111) found that the participant’s preferred language was significantly related to the MMSE score (p = 0.015) but not the RUDAS score (p value not reported).²² A third study (n = 204) reported an association between English as a first language and performance on the MMSE (p < 0.01) but not on the RUDAS (p = 0.30).²⁴ The RUDAS was also noted to be easier to translate between languages in one study, although this was not formally studied.²⁸

Immigrant status

One study (n = 137) looked specifically at the immigrancy status of participants²⁶ and showed that test performance on the RUDAS was not significantly affected (p = 0.08), whereas it was on the MMSE (p = 0.02).

Clinician preference

One study showed that clinician preference was generally stronger for the MMSE than the RUDAS (6 v. 1 of 12 clinicians preferred the measure, respectively), which was thought to be due to greater familiarity with the MMSE. Despite the novelty of the RUDAS, the authors commented that clinicians preferred the RUDAS in culturally and linguistically diverse populations in this study, although numerical estimates were not provided.¹⁹

Other results

Two studies examined test–retest reliability of the RUDAS and found it to be quite high (ICC 0.98 and 0.96).^10,20 Interrater reliability was reported in 2 trials and was also found to be high (ICC 0.99 and 0.71).^10,27 Two studies showed good internal consistency of the RUDAS (Cronbach α = 0.89 and 0.74).^20,23

Interpretation

The RUDAS has been studied in many parts of the world and is a useful brief tool to both rule in (positive likelihood ratio 5.5) and rule out (negative likelihood ratio 0.27) dementia. It has a particularly high specificity across cultures in immigrant populations (91.4%). The RUDAS performed similarly to the MMSE, and results may be less affected by education and language. The effect of immigrancy status on results of the RUDAS needs further study. There is also a suggestion from one study that clinicians appear to prefer the RUDAS to the MMSE for use in diverse populations. The RUDAS thus appears to be a useful cognitive assessment tool for all patients with suspected impairment, and may be particularly preferable in patients from culturally and linguistically diverse populations.

It has been well-established both in the literature^8,9 and in clinical practice that many available cognitive assessment tools have cultural and educational biases, including the MMSE. The RUDAS was designed to address these limitations and aimed to better assess patients from diverse backgrounds. It is important to note that although the RUDAS was interpreted independently of the formal diagnosis of cognitive impairment in most of the included studies, the MMSE was not. The studies would therefore be biased toward showing stronger psychometric properties of the MMSE. Despite this potential bias, the RUDAS still performed similarly to the MMSE in these studies. This further suggests that the RUDAS is a useful tool for cognitive assessment.

Strengths and limitations

We performed a comprehensive review of the literature through an extensive search of multiple databases, reviewed all bibliographies, and attempted to contact all corresponding authors. We are thus confident that all relevant articles have been identified. The overall quality of the articles that assessed the psychometric properties of the RUDAS were strong. The diversity of countries that have studied the RUDAS and the strong test performance both between and within cultures also add to the generalizability of the results. The population in the various studies is also similar to those seen in most outpatient clinical settings.

There were several limitations to our review. No studies have analyzed the responsiveness of the RUDAS to cognitive changes over time or with any specific interventions such as a trial of medications. As mentioned above, only 2 studies investigated test–retest reliability data, interrater reliability or internal consistency of the RUDAS. One further limitation is that all studies used formal interpreters, which may not always be available in clinical practice; this potentially limits generalizability when testing must be done in a language that is not the patient’s primary language. We were also unable to obtain full data on education level and literacy for several studies (Table 2) despite attempts to contact study authors. Finally, 3 studies published in abstract form could not be included because we were unable to obtain full data. Inclusion of the data from the one abstract with sufficient information to perform a sensitivity analysis²⁴ showed no significant change in the reported sensitivity and specificity of the RUDAS (Appendix 3, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140802/-/DC1).

Conclusion

The RUDAS is a freely available, effective brief cognitive assessment tool that has shown strong psychometric properties in several countries. It shows particular advantage in culturally and linguistically diverse populations. Given our aging immigrant population in Canada and the anticipated increase in dementia prevalence worldwide in coming decades, earlier and more accurate detection of dementia in these populations will become increasingly important.

Further studies to evaluate the effect of education, language and immigrancy status on RUDAS performance are needed. Research focusing on validating this tool in Canada’s diverse populations should be undertaken.