Can Rowland Universal Dementia Assessment Scale (RUDAS) Replace Mini-Mental State Examination (MMSE) for Dementia Screening in a Thai Geriatric Outpatient Setting?

Abstract

Mini-mental-State Examination (MMSE) is a widely used tool for dementia screening. However, several limitations are found and the Rowland Universal Dementia Assessment Scale (RUDAS) appears to be an alternative test. The objective in this study was to compare the performance of MMSE-Thai 2002 and RUDAS-Thai for dementia screening, and to determine their performances and identify their optimal cut-off points. The participants were older patients from a Geriatric and Neurology Outpatient Clinic, Srinagarind Hospital, Khon Kaen University. The RUDAS-Thai and the MMSE-Thai 2002 were administered to each participant. Subsequently, a specialist physician assessed each participant for dementia. Results showed the area under receiver operating characteristic curve for both RUDAS-Thai (81.0%; 95%CI, 74.8-87.2) and MMSE-Thai 2002 (81.2%; 95%CI, 74.9-87.4) were equal. A score of 24 or lower provided an optimal cut-off point. Our finding supports that the RUDAS-Thai can be an effective alternative test for dementia screening. For both test scores, a score of 24 or lower is an optimal cut-off point to provide an indication of developing dementia.

Introduction

Dementia is an acquired syndrome of decline in multiple cognitive abilities, including memory, thinking, behavior, and emotion, severe enough to interfere with daily function. ¹ It is estimated to affect more than 13% of the population aged >71 years or older. ¹ As ageing is one of the important risk factors for developing dementia and the number of older people in the population have been increasing as well as life span increases, the global prevalence of people with dementia could arise. The prevalence of people with dementia was estimated as 24 million in 2001 and 35 million in 2010. A recent study has predicted prevalence to be approximately 50 million people in 2020 with two thirds living in developing countries. ² This syndrome has a tremendous physical, emotional, and financial impact on the patient, the family, and society. ¹

Currently, the diagnosis of dementia is mainly based on clinical judgment. Screening for cognitive impairment in primary care settings is controversial. Given the higher prevalence of dementia in secondary and tertiary care settings, high number of undiagnosed patients, and new developing available treatment, early recognizing of dementia is crucial. ^{2 –6}

The most commonly used and studied test, which has acceptable accuracy is the Mini-Mental State Examination (MMSE). ^{2,4,7 –9} Its sensitivity varies from 71% to 92% and specificity ranges from 56% to 96% depending on the cutoff point used for an abnormal test. Other screening tests are promising and need further study. ³ The Thai version of the MMSE is MMSE-Thai 2002 which was validated in people aged >60 years. It is currently the most commonly used tool and provides different cutoff points based on educational level. ¹⁰ The sensitivity in illiterate people, 6 years of education or lower, and higher than 6 years of education was 35.4%, 56.6%, and 92%, respectively. The specificity of this test among these people was 76.8%, 88.9%, and 91.2%, respectively. The MMSE-Thai 2002 takes about 10 to 20 minutes to be administered in 50% of administrators. ¹⁰ Basically, the MMSE was developed in an English-speaking population, with versions in other languages mostly using direct translation rather than a culturally specific edition; therefore, some questions within the MMSE are difficult to understand and several limitations of the MMSE including educational level, cultural background, age, language, and some important cognitive domains, such as frontal lobe function, have not been assessed. ^{9 –12}

The Rowland Universal Dementia Assessment Scale (RUDAS) is a 6-item questionnaire, developed within a multicultural community in Australia. It takes only 10 minutes to administer. Items address a variety of cognitive domains including memory, praxis, visuoconstruction, language, and visuospatial domain. ^{12 –15} Therefore, it assesses frontal lobe function which is not assessed in commonly used tools. It provided sensitivity of 89% and specificity of 92% at a cutoff point of 23 of 30. The area under the receiver–operating characteristic (ROC) curve (AUC) for the RUDAS was 0.94 (95% confidence interval [CI] 0.87-0.98). At a cutoff point of 23 (maximum score of 30), sensitivity and specificity were 89% and 98%, respectively. Interrater and test–retest reliabilities were 0.99 and 0.98, respectively. ^12,13,15 However, RUDAS-Thai revealed less sensitivity and specificity than the English version at the same cutoff point. ¹⁶ It did not appear to be influenced by language and gender. It could be translated directly into another language without changing any format of any of the items, ^12,13,15 even if translated version into Malayan and Thai did show the influence of education level on the test performance. ^14,16 The RUDAS correlated well to MMSE (Spearman’s coefficient of .85). ¹² According to the area under the ROC curve, it appeared that the performance of both RUDAS (0.82-0.92) and MMSE (0.83-0.91) was equally accurate in diagnosing dementia. ^12,17

The RUDAS-Thai appears to avoid some limitations, the MMSE-Thai 2002, and can possibly be an alternative test for dementia screening in outpatient geriatrics. Thus, the primary aim of this study was to compare the test performance between the RUDAS-Thai and the MMSE-Thai 2002. The secondary objective was to demonstrate both test performances on different cutoff points and identify their optimal cutoff points.

Materials and Methods

This is a substudy of the performance of the RUDAS for cognitive screening in a geriatric outpatient setting which was a cross-sectional study. ¹⁶ Data collection was carried out between September 2010 and March 2011 at the Geriatric and Neurology Clinic, Faculty of Medicine, Khon Kaen University, Thailand.

Participants

This study consisted of 200 elderly patients from the Geriatric Clinic and Neurology Clinic of Srinagarind Hospital. Most of the patients were referred for problems associated with physical frailty, such as falls and gait instability and various neurological conditions. Approximately 20% of the patients were referred because of cognitive impairment. Clinical assessment, physical examination, and standard routine care were performed to all the participants. Factors that could influence performance including vision, hearing, depression, anxiety, musculoskeletal disorders, fatigue, dysarthria, and dysphasia were assessed and further investigated if indicated. The frequency of follow-up depended on the physicians’ assessment.

The inclusion criteria were Thai-speaking participants, age >60 years, they and/or their proxies were willing to participate in the study, and there was no apparent acute illness that could affect the performance of the test. The participants who were reluctant to complete the tests, unable to understand Thai or local language, and lost to follow-up with a geriatrician and a neurologist were excluded.

Instrument

The instrument used in this study was the RUDAS-Thai. The original 6-item version of RUDAS was translated directly into Thai language, and the content was validated by a professional translator. The RUDAS-Thai was scored by a single rater. Test–retest reliability of the RUDAS was assessed prior to the main study by giving scores to the same participant from the video recorder 1 week apart.

Procedure

A trained member of the aged care team administered the RUDAS-Thai and the MMSE-Thai 2002in a random order at the Geriatric or Neurology clinic. Dementia was assessed by a Geriatrician or a Neurologist, according to Diagnostic and Statistical Manual of Mental disorder (Fourth Edition, Text Revision) (DSM-IV-TR) criteria within 1 to 2 weeks after the RUDAS-Thai and the MMSE-Thai 2002 was performed. In addition, Clinical Dementia Rating was used to assess the disease severity. The Barthel Activities of Daily Living (ADL) index and the Lawton ADL score were used to assess daily function. ^{18 –20} Further investigations were organized if required. Trained member of the aged care team and the geriatrician and neurologist were blinded of the results of each other. Information about the demographic characteristics of elderly person and his or her informant were also collected.

Sample Size Calculation

Sample size calculation was based on the area under the ROC curve (AUC) according to the methodology of Hanley and McNeil. ²¹ The ROC curves were used to summarize the accuracy of diagnostic tests. This method varies with the sample size until a sufficiently small standard error (SE) of the AUC was achieved. Because of the complexity of the formula, a Web-based calculator (www.anaesthetist.com/mnm/stats/roc/#stderr) was used to determine the SE. Finally, a sample size of 200 participants was adequate and feasible to conduct in clinical practice at the AUC of 0.8 and SE of 0.044.

Statistical Analyses

Baseline characteristics were summarized using frequency and percentage for categorical variables. For continuous variables, mean and standard deviation (SD) were used. In case that the continuous variable was not normally distributed, median, minimum, maximum, and interquartile range were used instead.

The screening accuracy and determination of optimal cutoff point for dementia screening of the RUDAS-Thai and the MMSE-Thai 2002 was summarized by the sensitivity, specificity, predictive positive value, predictive negative value, Yuden index, AUC, and likelihood ratio. Youden index was used to determine an optimal cutoff point. The ROC curve was used to summarize the overall accuracy of the RUDAS-Thai and the MMSE-Thai 2002 for dementia detection. The AUC was estimated for each screening tool separately, along with their 95% CIs.

Regarding the cross-sectional sample in this study, it was often difficult to distinguish very mild dementia from mild cognitive impairment (MCI). ²² To minimize the misclassification bias, our analysis included only participants with normal cognition and those with dementia.

All the data analyses were performed by STATA version 10.0 (StataCorp, College Station, Texas).

Ethics approval was provided by ethics committee of Medicine Faculty, Khon Kaen University under the respect of Helsinki Declaration.

Results

Participant Characteristics

Two-hundred geriatric participants of the Internal Medicine Outpatient Clinic of Srinagarind Hospital medical school were enrolled. Of all, 89 (44.5%) participants had dementia, 89 (44.5%) had normal cognition, and 22 (11%) had MCI. In all, 178 participants were in primary analyses. Baseline characteristics were shown in Table 1. Compared with participants with normal cognition, those with dementia had lower educational level and more informant presence. Furthermore, participants with dementia were more likely to be dependent in activities of daily living based on the Barthel ADL index and the Lawton score, supporting a correlation of functional measure in dementia. The distribution of factors potentially affecting the performance of the test included visual and hearing impairment, psychiatric conditions, musculoskeletal disorders, and noncognitive neurological disorders. These were similar between the 2 groups. In the MCI group (11%), the mean age was 71.36 (SD 5.80), their educational level was distributed equally in primary school and higher than primary school, and the distribution of gender was equal.

Table 1.

Demographic Data of Participants

Characteristics	No dementia (N = 89)	Dementia (N = 89)	P value
Age (year; mean, SD)	70.18 (5.57)	71.90 (7.23)	.11
Female (%)	53 (59.55%)	37 (41.57%)	.02
Literacy and educational level (%)
Illiteracy	1.12%	7.87%
6 years or lower	42.70%	62.92%	<.001
More than 6 years	56.18%	29.21%
Informant present (%)	42.70%	78.65%	<.001
Barthel index (%)
Total dependence	0.00%	3.37%
Severe dependence	0.00%	10.11%
Moderate dependence	5.62%	8.99%	<.001
Mild dependence	94.38%	77.53%
Lawton score (median, Q1-Q3)	7 (6-7)	6 (3-7)	<.001
Numbers of factors potentially affecting performance (median, Q1-Q3)	0 (0-1)	1 (0-2)	.014
Depression	2 (2.24%)	2 (2.24%)	1.00
Anxiety	5 (5.62%)	4 (4.49%)	1.00
RUDAS-Thai (median, Q1-Q3)	26 (24-28)	21 (18-24)	<.001
CDR (%)
No dementia	75.28%	–	<.001
Questionable dementia	24.72%	34.83%
Mild dementia		26.97%
Moderate dementia		16.85%
Severe dementia		21.35%

Abbreviations: Barthel index, Barthel Basic Activities of Daily Living Scale (range 0-4, higher scores indicate better function); Lawton, modified Lawton Instrumental Activities of Daily Living Scale (range 0-7, higher scores indicate better function); RUDAS, Rowland Universal Dementia Assessment Scale; CDR, Clinical Dementia Rating Scale; Q1-Q3 refers to interquartile range; factors potentially affecting performance on the RUDAS included visual or hearing impairment, psychiatric disease (depression, anxiety, or psychosis), dysarthria or dysphasia, neurological disease, and musculoskeletal disorders.

Screening Accuracy and Determination of Optimal Cutoff Point for Dementia Screening

The performance of the RUDAS-Thai for dementia screening in geriatric outpatients is summarized in Table 2, where the RUDUS-Thai represents different cutoff points. The screening performance of the MMSE-Thai 2002 for dementia was summarized in Table 3.

Table 2.

The RUDAS Performance on Screening for Dementia According to its Various Cutoff Point

Cutoff points	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Youden index	AUC under ROC	LR+	LR−
<19	38.2	97.8	94.4	61.3	0.742	0.68	17.00	0.632
<20	48.3	96.6	93.5	65.2	0.449	0.725	14.30	0.535
<21	53.8	91	85.5	65.9	0.448	0.719	5.88	0.519
<22	61.8	84.3	79.7	68.8	0.461	0.73	3.93	0.453
<23	67.4	82	78.9	71.6	0.494	0.747	3.75	0.397
<24	78.7	60.7	66.7	73.8	0.405	0.702	2.00	0.345
<25	79.3	59.6	67	73.6	0.389	0.694	1.96	0.347

Abbreviations: RUDAS, Rowland Universal Dementia Assessment Scale; PPV, positive predictive value; NPV, negative predictive value; AUC under ROC, area under the receiver operating characteristic curve; LR+, likelihood ratio positive; LR−, likelihood ratio negative.

Table 3.

The MMSE-Thai 2002 Performance on Screening for Dementia According to its Various Cutoff Points

Cutoff points	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Youden index	AUC under ROC	LR+	LR−
<19	50.6	91	84.9	64.8	0.416	0.708	5.63	0.543
<20	56.2	87.6	82	66.7	0.438	0.719	4.55	0.5
<21	60.7	83.1	78.3	67.9	0.438	0.719	3.6	0.473
<22	67.4	76.4	74.1	70.1	0.438	0.719	2.86	0.426
<23	73	73	73	73	0.46	0.73	2.71	0.369
<24	78.7	66.3	70	75.6	0.45	0.725	2.33	0.322
<25	89.9	50.6	64.5	83.3	0.405	0.702	1.82	0.2

Abbreviations: MMSE, Mini-Mental State Examination; PPV, positive predictive value; NPV, negative predictive value; AUC under ROC, area under the receiver operating characteristic curve; LR+, likelihood ratio positive; LR−, likelihood ratio negative.

The ROC Curves

The AUC of the RUDAS-Thai (0.81, 95% CI: 74.8-87.2) was similar to the AUC of the MMSE-Thai 2002 (0.81, 95% CI: 74.9-87.4) as shown in Figure 1.

Figure 1.

The ROC curve of RUDAS-Thai and MMSE-Thai 2002 on screening for dementia. MMSE indicates Mini-Mental State Examination; ROC; receiver–operating characteristic curve; RUDAS, Rowland Universal Dementia Assessment Scale.

Correlation Between RUDAS-Thai and MMSE-Thai 2002

The RUDAS-Thai and MMSE-Thai 2002 scores correlated highly, with a Pearson’s coefficient of .80 (95% CI: 0.745-0.85, P < .0001).

Discussion

The performance of the RUDAS-Thai and the MMSE-Thai 2002 was analyzed in screening dementia in a geriatric outpatient setting. The results demonstrated similarity in the performance according to the AUC under ROC curve of 0.81. Both tests showed high correlation based on Pearson’s correlation coefficient. A previous study showed that the AUC for the RUDAS (0.92) was also similar to the AUC for the MMSE (0.91) and showed good correlation with a Spearman’s coefficient of .85 although their results illustrated higher values than our study. ¹² This difference could be explained by our primary study, where the performance of the RUDAS for cognitive screening in a geriatric outpatient setting showed that years of education influenced the performance of the RUDAS-Thai which differed from the original study as most participants therein had more than 6 years of education while the majority of our participants did not, supporting the study of a South Indian sample when the RUDAS was translated into Malayan. ^{14 –16} Other probable factors that could influence test performance include diverse cultural background and language of preference.

In our study, the RUDAS-Thai and the MMSE-Thai 2002 demonstrated good reliability and discriminative properties in dementia screening. According to Table 2, a cutoff point of 24 or lower yielded good dementia screening accuracy based on its performance including the Youden index which was different from the recommended cutoff point of 23 of 30. ^12,15 The screening performance of MMSE-Thai 2002 scores as showed in Table 3 also revealed that an MMSE-Thai 2002 score of 24 served as the optimal cutoff point for screening patients with developing dementia.

Our data suggest that using the RUDAS-Thai’s recommended cut-off points yields similar results to the MMSE-Thai 2002 for dementia screening in geriatric patients. Although the MMSE-Thai 2002 has become the most commonly used screening tool for dementia detection in Thailand, but given the limitations of the MMSE including the MMSE-Thai 2002, namely variability in sensitivity and specificity influence of educational levels, cultural background, age, language, and that it is time consuming and lacks some important cognitive domain assessment, ^10,11 the RUDAS-Thai could be an alternative test to administer in Thai geriatrics as only educational levels affected the test performance.

On the basis of our results, we recommend raising the RUDAS-Thai cutoff point of 24 of 30 in detecting development of dementia.

Some limitations were observed in our study. First, the prevalence and severity of dementia are likely to have more than expected in the general population which Srinagarind Hospital is a tertiary care hospital. Second, a misclassification bias could have occurred which was possible due to study design which lacked a longitudinal follow-up and brain pathology. Third, the diagnosis of dementia is primarily a clinical judgment without the convenience of an easily available biomarker. However, we excluded the participants with MCI in primary analysis and the diagnosis of dementia was based on DSM-IV-TR criteria which are widely used. Fourth, as most participants had 6 years or lower of education, the results probably better reflects the performance of the test in this group. Finally, as gender is one of known risk factors for dementia and there are significantly less males in the nondementia group, new well-balanced studies in terms of gender should be performed if possible.

Conclusion

The RUDAS-Thai is proposed as a reliable cognitive screening tool that can replace MMSE-Thai 2002 in Thai geriatrics. For both test scores, we recommend that a score of 24 or lower is an optimal cutoff point to indicate development of dementia.