Accuracy of Korean-Mini-Mental Status Examination Based on Seoul Neuro-Psychological Screening Battery II Results

Abstract

Background

The Korean-Mini-Mental Status Examination (K-MMSE) is a dementia-screening test that can be easily applied in both community and clinical settings. However, in 20% to 30% of cases, the K-MMSE produces a false negative response. This suggests that it is necessary to evaluate the accuracy of K-MMSE as a screening test for dementia, which can be achieved through comparison of K-MMSE and Seoul Neuropsychological Screening Battery (SNSB)-II results.

Methods

The study included 713 subjects (male 534, female 179; mean age, 69.3±6.9 years). All subjects were assessed using K-MMSE and SNSB-II tests, the results of which were divided into normal and abnormal in 15 percentile standards.

Results

The sensitivity of the K-MMSE was 48.7%, with a specificity of 89.9%. The incidence of false positive and negative results totaled 10.1% and 51.2%, respectively. In addition, the positive predictive value of the K-MMSE was 87.1%, while the negative predictive value was 55.6%. The false-negative group showed cognitive impairments in regions of memory and executive function. Subsequently, in the false-positive group, subjects demonstrated reduced performance in memory recall, time orientation, attention, and calculation of K-MMSE items.

Conclusion

The results obtained in the study suggest that cognitive function might still be impaired even if an individual obtained a normal score on the K-MMSE. If the K-MMSE is combined with tests of memory or executive function, the accuracy of dementia diagnosis could be greatly improved.

INTRODUCTION

Dementia incorporates a range of deficits, which include executive dysfunction, impaired judgment, and varying degrees of memory loss. This is particularly important when defining dementia, since memory loss is not the sole factor precluding dementia diagnosis. In order to diagnose dementia, testing for the presence of cognitive impairment and other factors that impact on daily on the daily routine should be performed comprehensively, as these are equally important.1) Due to economic limitation, environmental and time factors many of the elderly have not received professional dementia evaluation, and the subsequent support that accompanies this. Therefore, a cognitive screening tool that could quantify overall cognitive function and be easily implemented within a short time would be highly beneficial.2) A screening test for dementia that can be performed easily in primary medicine at outpatient clinic, the Mini-Mental Status Examination (MMSE) is currently used all around the world in clinical and community settings.3,4)

At first, the MMSE was not developed for the screening of dementia. It was originally designed as a simple tool to assess a patient's cognitive function whilst in hospital. However, in some studies, its reliability and validity suggested that the MMSE might be equally effective in the screening of dementia, in particular, for Alzheimer's disease.1,5) In Korea, the South Korean version of the MMSE such as the as the Korean-Mini Mental State Examination (K-MMSE) and Mini Mental State Examination-Korean version (MMSE-K) have been developed and widely used.6,7)

In the MMSE-K, it is suggested that a score over 24 points is 'definite normal,' while below 19 points is 'definitive dementia,' and 20–23 points is 'suspicion of dementia.' If the patient is illiterate, it was suggested that a score be determined by adding four points to the score for features such as disorientation (one point), attention (one point), concentration (two points), and language disorder (one point). However, since the South Korean higher education level is gradually increasing, and some limitations were identified by comparative study with data from foreign countries, the K-MMSE was created and standardized in 1998. Currently, it is widely used to inspect and diagnose dementia patients. K-MMSE scores are heavily influenced by education and age, so with a diagnosis of dementia, no single score is used. The score is therefore determined by applying the criteria with regard to the level of education and each age.1)

The K-MMSE question set is relatively easy and it is likely to produce 'false negative' responses by the ceiling effect in people of slight cognitive impairment.8) Therefore, when using a K-MMSE to diagnose dementia, false negative rates may reach 20%–30%, as suggested in previous studies.6) However, the above-mentioned false negative rate was analyzed before the specific criteria of the K-MMSE were created, so this score did not account for age or educational background. Additionally, the previous level for detection of cognitive impairment was set at below 24 points by the basis of the foreign research results, which caused restrictive issues.9,10,11) When compared with comprehensive neuropsychological tests such as the Seoul Neuropsychological Screening Battery (SNSB), data from K-MMSE tests that applied the criteria according to education and age derived false negative rate results over 19.7%.12)

In recent years, the SNSB has been modified and complemented to produce the SNSB-II.13) The SNSB-II expands the age criteria to 45 years old to 90 years old from its previous range of 55 years to 80 years of age. Further to this, new forms of assessment have been added to reflect the recent trends in research. Based on nationwide large-scale sample (1,100 people), it became possible to provide an indication, such as cognitive domain scores, using new psychometric criteria.

This study will aim to examine the accuracy of the K-MMSE as a dementia-screening test, with a particular focus on the incidence of false negative and positive results. The K-MMSE test will be adjusted in accordance with the age and educational background of subjects, in addition to the results of comprehensive neuropsychological tests based on the SNSB-II.

METHODS

1. Subjects

All subjects were neurology outpatient or inpatients that consulted with neurological problems between December 2012 and January 2015 in the Gwangju Veterans Hospital. Each subject featured suspected cognitive impairment and received evaluation using the K-MMSE and SNSB-II. In total, 713 subjects were analyzed (male 534 and female 179), with an average age of 69.3±6.9 years and 8.1±4.3 years of education.

2. Method of Study

The K-MMSE and SNSB II were used to assess all patients. The K-MMSE incorporates a range of elements that include time orientation (5 points), spatial orientation (5 points), memory registration (3 points), attention and calculation (5 points), memory recall (3 points), language (16 points), and space-time configuration (1 point). This creates a total score of 30 points.13) The SNSB-II evaluates four cognitive factors such as memory, language skills, space-time functions, and executive function. In this study, the evaluation of each cognitive domain conformed to the criteria of SNSB-II. Memory was evaluated by delayed recall using Seoul Verbal Learning Test (SVLT) and the Rey Complex Figure Test (RCFT). Language skills were assessed by naming (Korean-Boston Naming Test, K-BNT), and time and space function was evaluated using the RCFT. Executive function was evaluated using a variety of tests, including the motor control and perseveration test (motor impersistence, contrasting program, go-no-go, fist-edge-palm, alternating hand movement, alternating square and triangle, and Luria loop) the controlled word association test (COWAT) and the Stroop color reading forward reaction of Korean-Color Word Stroop Test.

The cut-off score 'average score-1 standard deviation (15 percentile)' for each test was established by using the age and education criteria presented in SNSB-II to evaluate cognitive function. The scores of the K-MMSE test and SNSB-II result were divided into normal and abnormal in 15 percentile standards. In addition, the motor control and perseveration tests were divided into normal and abnormal based on the SNSB-II criteria.13)

For SNSB-II assessment, area-specific disability criteria were defined as follows, in accordance with the standards of cognitive dysfunction defined by SNSB-II. (1) Memory: at least one is abnormal in A and B. A (Verbal memory): SVLT delayed recall scores are abnormal (<15%). B (Visual memory): RCFT delayed recall scores are abnormal (<15%). (2) Language skills: K-BNT (full) is abnormal (<15%). (3) Space-time function: the RCFT copy is abnormal (<15%). (4) Executive dysfunction: two or more are abnormal in A, B, and C. A (Motor examination): three kinds or more are abnormal in the following tests: motor impersistence, contrasting program, go-no-go, fist-edge-palm, alternating hand movement, alternating square & triangle, and Luria loop. B (COWAT): at least one or more is abnormal in the following three tests: animal, supermarket, and Hangul consonants (ᄀ, ᄋ, and ᄉ). C: Stroop color reading forward reaction is abnormal (<15%).

In this study, cognitive function was assessed in the areas mentioned above, and then classified into groups, depending on the incidence and number of impairments. This was then cross-referenced according to the definition of dementia and the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition. 14) After subjects were classified into groups of cognitive impairment, they were assessed using the K-MMSE to evaluate the accuracy of this test. The K-MMSE was assessed for sensitivity, specificity, positive predictive value, negative predictive value, false positive rate, and false negative rate. Statistical analysis took place using the Korean version of Microsoft Office Excel 2007 (Microsoft Office Co., Redmond, WA, USA). A P-value less than 0.05 was accepted as significant.

RESULTS

1. Comparing Results of the K-MMSE and SNSB-II

The results of the K-MMSE were compared with the results of the SNSB-II (Table 1). Among the 713 patients, 480 patients (67.3%) featured a 'normal' K-MMSE score, and 233 patients (32.7%) had an 'abnormal' K-MMSE score. Within the 'normal' category of the K-MMSE score, 267 patients (55.6%) were classified as part of the 'no cognitive impairment' group by the SNSB-II, while the remaining 213 patients (44.4%) were classified as part of the 'cognitive impairment' group. Further to this, in the 'abnormal' category of the K-MMSE score (233 patients), 203 patients (87.1%) were classified as part of the 'cognitive impairment' group by the SNSB-II, while 30 patients (12.9%) were classified as part of the 'no cognitive impairment' group.

Table 1. A comparison of the K-MMSE with the SNSB-II.

K-MMSE, Korean-Mini-Mental Status Examination; SNSB, Seoul Neuropsychological Screening Battery.

If the SNSB II is a reliable and accurate method of assessing cognitive impairment, this would suggests that the sensitivity of the K-MMSE was 48.7%, the specificity of K-MMSE was 89.9%, the false-positive rate was 10.1% (30/297), and the false negative rate was 51.2%. While the positive predictive value of the K-MMSE was high as 87.1%, the negative predictive value of K-MMSE was 55.6%. Therefore, up to 44.4% of patients assigned to the 'normal' category by the K-MMSE may actually have belonged to the 'cognitive impairment' group.

2. Characteristics of the K-MMSE False-Positive and False Negative Groups

Patients featuring a 'normal' category of K-MMSE score were classified as part of the 'no cognitive impairment' group and 'cognitive impairment' group (false negative group) by SNSB-II results. The average age of the 'no cognitive impairment' group was 68.3±7.1 years, with the average age of the 'cognitive impairment' group at 69.7±7.2 years. A statistically significant difference was found between the two groups (P=0.035). In addition, the average years of education in each category was 7.6±4.1 and 8.1±4.3 years, respectively, with no statistically significant difference found between the two groups (P=0.180) (Table 2).

Table 2. Comparison of no cognitive impairment with cognitive impairment in the MMSE normal and abnormal group.

MMSE, Mini-Mental Status Examination.

Patients classified within the 'abnormal' category of K-MMSE scores were assigned to both the 'no cognitive impairment' group and 'cognitive impairment group' by SNSB-II results. The average ages in each category were defined as 68.4±6.4 and 70.1±6.3 years, respectively, while the average years of education in each category were 9.2±4.1 and 8.7±4.5 years. No statistically significant difference was found between the two groups for either average age or years in education (each P=0.172, P=0.485) (Table 2).

The data obtained from patients with K-MMSE scores in the 'normal' range but classified as 'abnormal' by SNSB-II (false-negative population, n=213) was examined more closely to determine exactly which category of impairments the K-MMSE failed to detect. The results of the SNSB-II were classified into either the 'no cognitive impairment' group or 'cognitive impairment' group by the criteria 'memory disorders+one or more other cognitive impairment.' Therefore, all patients belonging to the 'cognitive impairment' group had a memory disorder. In addition to memory disorders, the occurrence of executive dysfunction (87.8%), language disorders (31.5%), and space-time configuration disorders (27.2%) accompanied cognitive dysfunction.

Patients with K-MMSE scores in the 'abnormal' range but classified as 'normal' by SNSB-II (false-positive group, n=30) were more closely examined with regard to their performance on items of the K-MMSE. Reduced performance was shown in memory recall (60.0%), time disorientation (53.3%), attention and calculation (46.6%), language skills (0.17%), memory registration (0.13%), space-time configuration (0.13%), and spatial disorientation (0.07%).

DISCUSSION

As stated in the opening paragraph, MMSE was not initially developed for dementia screening; it was created as a simple tool for examiners to check the overall cognitive function of patients. Given its ease of use, the MMSE became a popular tool for the diagnosis of dementia in a variety of settings.

According to results of this study, when K-MMSE is used as a method of assessing cognitive impairment, a patient with a K-MMSE score in the 'normal' range is may be equally likely to have received a false positive result, with a negative predictive value of 55.6% for the K-MMSE test. Patients with a K-MMSE score in the 'abnormal' range were less affected, with a positive prediction value of 87.1%.

The performance of specific cognitive domains within false-negative groups was examined to determine which areas of cognitive function the K-MMSE was weak at screening. Memory disorders (100%), executive dysfunction (87.8%), language disorders (31.5%), and spacetime configuration disorders (27.2%) were detected in the false-negative group. Several previous studies including the SNSB, identified that the MMSE did not adequately evaluate executive function.12,15,16)

According to this study, many patients within false positive groups have a low score in K-MMSE performance areas such as memory recall (60.0%), time orientation (53.3%), intention, and calculation (46.6%). In previous study based on SNSB, the majority of false-positive group also received a low score at language ability category,12) but in this study only 0.17% patients did. In a previous study based on SNSB, K-MMSE was performed in patients with a low level of education. In this instance, there was a high possibility that patients with no cognitive impairment could be misjudged as having a cognitive impairment, 12) but in the current study, no significant results were detected for this effect. Since a lack of education was believed to affect a patient's K-MMSE score, in the current study, a correction of '100-7' to '40-3' items in the calculation of K-MMSE score was undertaken, in order to reduce the false-positive rate among patients.

This study has a few limitations. Since the study was limited to patients from one hospital, this reduced the diversity of subjects available for the study. As this is not necessarily a reliable indicator of patients in other regions, it may be difficult to generalize the above results.

This study assessed the accuracy of K-MMSE by comparison with SNSB-II, and in doing so, identified weaknesses in the test that could potentially compromise its ability to diagnose cognitive impairment. The high incidence of false negative results suggests a relative inaccuracy as a diagnostic technique, and therefore potentially depriving numerous patients of suitable care. For future use in dementia diagnosis, the accuracy of K-MMSE testing might be improved by incorporating simultaneous measures of memory and executive function, to more finely assess areas of cognitive impairment that might be missed by the K-MMSE.