Does the Abbreviated Mental Test Accurately Predict Cognitive Impairment in Thai Older Adults? A Retrospective Study

Abstract

The Abbreviated Mental Test is a screening tool for cognitive impairment in older adults now used in Thailand’s annual national cognitive assessment required for all community-dwelling older adults in Thailand, however its validity has not been established for this purpose. This retrospective study evaluated the results of this abbreviated test as well as the Mini Mental State Examination, in a sample of older adults who participated in the national cognitive assessment in 2018 in Udon Thani Province, Thailand. Of the 174,227 cases, 1518 cases had scores on both tests. The Mini-Mental State Examination is performed on a select group of individuals who scored low on the initial and briefer test.

Results indicated that the proportion of cognitive impairment as defined by the Abbreviated Mental Test (87.7%) was five times higher than indicated by the more rigorous Mini-Mental State Examination (16.3%). At the AMT standard cut point of 8 out of a total possible score of 10 for the presence of cognitive impairment, sensitivity was low (12.8%) but specificity was high (90.3%), while the optimal cut-point of 5 on the Receiver Operating Characteristic Curve errs on the side of being highly sensitive but not specific enough to screen for cognitive impairment. Additionally, one item on the AMT was answered incorrectly by 95% of responders. Although brief and easy to administer, the AMT may be a suboptimal choice for screening for cognitive impairment. When administered by volunteer health workers, the AMT may also possess limited reliability and validity. Cognitive screening administered by nurses should be considered to help detect cognitive impairment older adults dwelling in the community.

Background

Many countries around the world are facing a rapid growth of aging populations, with the fastest growth rates in developing countries.¹ With this rise in older adults aged 60 and over, there is a concomitant rise in the incidence of cognitive impairment and other disease conditions.² Even mild cognitive impairment may progress to dementia in some individuals which make early detection of cognitive impairment the objective of population screening measures in many countries to identify ‘at-risk individuals’ who may benefit from further evaluation and intervention.^{3, 4}

As with other countries, an aging population is of great concern in Thailand. In 2018, Thailand had a total population of 66.4 million people, of which 10.6 million (16.1%) were ≥60 years.⁵ The number of Thai older adults with dementia is predicted to increase from 617,000 in 2016 to 1,350,000 in 2037.⁶ In the past decade, a variety of Thai government initiatives have been undertaken to provide for long-term care of older adults with dementia, particularly community-based programs and health-promoting hospitals at the primary care level.⁷ To facilitate the identification and referral of individuals ‘at risk’ for dementia, the Thai Ministry of Public Health in 2015 initiated the use of two cognitive screening tests, the AMT and MMSE, to be used in a standardized annual screening process for all Thai citizens age ≥60 years. The standard screening process involves an initial assessment with the AMT by volunteer health workers in the community. AMT scores <8 suggest abnormal cognitive function and prompt the administration of the MMSE by nurses or public health officer in the sub-district health-promoting hospital.^8,9

Given that screening for cognitive impairment is newly required in Thailand, it is critical to assess the value of such screening methods. Since Thailand does not now have a clinical interview and diagnosis of mild cognitive impairment or dementia by a physician at the primary care level, our research question is: Does the AMT accurately predict cognitive impairment in community-dwelling older adults in Thailand? Therefore, we undertook a retrospective study which investigated the national screening tools and methodology for determining the incidence of cognitive impairment in Thai older adults.

Study Objectives

The objectives of this study were to 1) investigate the national cognitive screening tools by examining the relative sensitivity and specificity of the AMT (using the MMSE Thai 2002 as the “reference standard”) in a large sample of older adults in Udon Thani Province of Thailand, and 2) assess the methodology for determining the incidence of cognitive impairment in older adults living in the community.

Methods

Design:

A retrospective descriptive design was used. Secondary data analysis of a data set created from two merged data sets of Program Aging Survey of Udon Thani and the Health Data Center Dashboard from the Ministry of Public Health, Thailand in 2018 was conducted. We used data from this province because Udon Thani Provincial Public Health Office created its own merged database from these two national sources which had scores for both individual items and the total score for both the AMT and MMSE. In contrast, national-level data in the Health Data Center Dashboard reported only the number of older adults that had normal or abnormal cognitive function without the item-level or total scores from the AMT and MMSE.

Sample and setting:

In 2018, there were 181,952 older adults aged >60 in the Program Aging Survey of Udon Thani. Of these, 174, 227 had the data verified (95.75%) and 1518 cases were identified as cognitively impaired by the AMT. The second level screening with the MMSE was also available on these 1518 cases.

Screening tools:

Two instruments were used in this study; the AMT Thai version and the MMSE-Thai version. The original version of the AMT was developed by Hodkinson¹⁰ in 1972, and designed to quickly screen for cognitive impairment in older adults hospitalized patients in England. Since that time, the AMT has been widely used in many countries such as Australia,¹¹ Italy,¹² Spain,¹³ Iran,¹⁴ Poland,¹⁵ Singapore,¹⁶ and Hong Kong.^{17, 18} The original MMSE was developed by Folstein et al¹⁹ and widely used as a measure of cognitive impairment. The MMSE had been translated to many foreign languages such as Croatian, French, German, Dutch, Spanish for the US, Spanish for Latin America, European Spanish, Hindi, Russian, Italian, Bangla, Finnish, Gujarati, Sinhalese, Malay, Turkish Portuguese, Simplified Chinese and Thai.^19–22 Relevant information about these measures is described below.

The AMT Thai version:

This version was translated from the AMT original English version by the Department of Medical Service, Ministry of Public Health, Thailand.^8,9 Although there have been several publications in recent years reviewing the use of various cognitive screening tools for older adults in Thailand such as the Montreal Cognitive Assessment-Basic (MoCA-B),²³ the Thai Mental State Examination (TMSE) and the Mini-Mental State Examination (MMSE )²² and the Thai version of Mini-Cog,²⁴ there is no published study about the AMT use in Thailand.

The AMT original English version included 10 questions, with a 10-point scale. Each question scores 1 point, and the total score ranges from 0 (no correct answer) to 10 (10 correct answers). Administration of the test takes around 5 minutes and examines concepts of orientation (current time, address, current year, current location and recognition of two individuals), remote memory (age, date of birth, and year of a major national event), recent memory (remember an object), general knowledge and memory (current national figure like Prime Minister, King), and attention (count 20 backward to one). In the AMT Thai version, three items asked different questions compared to the British version; the item of “the year of first world war” was changed to “the year of great sorrow – for 1973 that had political and popular uprising between military and university students”.²⁵ The item of monarch was changed to “name of the present king” and “the address for recall” was changed to “current address.” The cut-off score on the AMT Thai version is ⁸. Based on the national standards derived from studies in other countries, older adults with AMT scores between 8–10 were classified as cognitively normal while those with score < 8 were identified as possible abnormal cognitive function.⁸

The Mini-Mental State Examination Thai version

The Mini-Mental State Examination Thai version (MMSE-Thai 2002) was adapted from the original version in English by Folstein et al.²⁶ and has been validated by the Department of Medical Service, Ministry of Public Health, Thailand.²² It is a 30-point questionnaire used extensively in clinical and research settings to measure cognitive impairment. Administration of the test takes between 5–10 minutes and examines functions including registration (repeating named prompts), attention and calculation, recall, language, ability to follow simple commands, and orientation. The cut-off score on the Thai-MMSE 2002 was adjusted based on participants’ educational levels. For example, 14 points out of 23 is the cut-off score for older adults who are illiterate or have not completed elementary school, 17 out of 30 is the cut-off score for those who have completed elementary education, and 22 out of 30 is the cut-off score for those who have completed levels of education higher than primary.²² Currently, the ‘reference standard’ measure in Thailand for cognitive impairment is the MMSE-Thai 2002, which uses the Ministry of Public Health recommended cut-off scores to determine cognitive impairment: a score of 14 for individuals with less than an elementary education, a score of 17 for individuals who have completed elementary education, and a score of 22 for those with some college or higher education.²² A higher score indicates better cognition and scores at or below the cut-off score are interpreted as abnormal cognitive function. We used the MMSE-Thai 2002 as the reference standard for comparison to evaluate the sensitivity and specificity of the AMT.

Data Analysis:

All statistical analyses were conducted using Stata (version 16) software.²⁷ Both AMT and MMSE Thai 2002 data showed non-normal distributions, as indicated by the Shapiro-Wilk (S-W) test. Descriptive statistics for these continuously scaled variables, therefore, include means, standard deviation (SD), and also medians, and interquartile range (IQR). We used the Bland and Altman levels of agreement methods and plots to characterize agreement between the AMT and MMSE scores after standardizing both scores to a percentage correct,²⁸ and Lin’s concordance to characterize the association overall.²⁹ Receiver operating characteristics curves (ROC) were created to evaluate how different cut-off scores on the AMT for a diagnosis of cognitive impairment performed relative to the reference standard diagnosis of an MMSE-Thai 2002 score.³⁰ Sensitivity, specificity, positive and negative predictive values, and area under ROC (AUC) were reported.

Ethical Considerations:

The study was approved by the Institutional Review Board of the Udon Thani Provincial Public Health Office to do a secondary analysis of the Udon Thani database (Approval number UDREC 0862). All data were de-identified prior to analysis.

Results

Demographic Information:

The characteristics of the sample are shown in Table 1. The average age of the sample was 72.87 years (SD=8.57), and the majority had an elementary education (89.3%). Cognitive impairment as defined by the AMT was 87.17% in the entire sample. The average score of AMT was 6 (mean 6.06, SD=1.96; median=7.00, IQR= 5.00-7.00). Using the MMSE-Thai 2002 cutoffs noted above, 16.3% of the sample was cognitively impaired. Males (Mean =6.22, SD=1.81) had a better cognitive function than females (Mean=5.97, SD=1.87) (t=2.56, p< .01). The average age of males and females was almost the same. Age-related cognitive decline was found in both the AMT and MMSE screening tests.

Table 1.

Minimum, maximum, mean, standard deviation, median, interquartile range, and percentage of age sex and education and cognitive impairment screening score of the older adults (N=1518)

Characteristics	N =1518
Age: Mean (±SD)	72.87 (±8.57)
Minimum – maximum	60-106
Gender: Female	968 (63.8%)
Male	550 (38.2%)
Education: Not completed elementary school	90 (5.9%)
Elementary school	1355 (89.3%)
Higher than elementary education	73 (4.8%)
Cognitive impairment screening score by AMT	1331 (87.17%)
Mean (±SD)	6.06 (± 1.96)
Median (±IQR)	7.00 (5.00–7.00)
Cognitive impairment screening score by MMSE	247 (16.3%)
Mean (±SD)	21.59 (±5.58)
Median (±IQR)	22 (±3)

The proportion of cognitive impairment as defined by the AMT (8 7.7%) was five times higher than indicated by the Mini-Mental State Examination (16.3%).

Abbreviated Mental Test:

Of 1518 cases, the median AMT was 7.00 (interquartile range=5.00–7.00). The average AMT score was 6.06 (SD=1.96). Item number 8 on the AMT “about the year of the great sorrow” was incorrectly answered by both the impaired group (96.1%), and the non-impaired group (94.7%). The items that almost half of older adults who were impaired could not answer were item 4 “The current year” (47%), item 7 “Date of birth” (56.5%), item 9 “Name of the present king” (55.8%) and item 10 “Count backward from 20 to 1” (69.8%) (Table 2).

Table 2.

Percent of the incorrect answer in AMT

AMT	AMT < 8 (N=1331)	AMT 8–10 (N=187)	Total (N=1518)
	incorrect (%)	incorrect (%)	Incorrect (%)
1. Age	19.1	2.7	17.1
2. Current Time	29.5	2.7	26.2
3. Address	13.5	6.4	12.6
4. Current Year	47.0	8.6	42.3
5. Current Location	16.4	12.8	15.9
6. Recognition of 2 persons	18.2	19.3	18.3
7. Date of Birth	56.5	22.5	52.3
8. Year of Great Sorrow on October 14	96.1	94.7	95.9
9. Name of the present king	55.8	8.0	49.9
10. Count backward from 20–1	69.8	17.6	63.4
Mean (±SD) score	6.06 (± 1.96)
Median (±IQR) score	7.00 (5.00–7.00)

Mini-Mental State Examination:

The median MMSE was 22.00 (interquartile range 19.00–25.00). The average MMSE score was 21.59 (± 5.58). Cognitive impairment as defined by MMSE-Thai 2002 was 16.3%. The most poorly performed item in MMSE was attention/calculation which was related to either completing serial subtractions mentally or spelling backward (10.8% correct).

Agreement between AMT and MMSE scores:

The Bland and Altman levels of agreement ²⁷ are shown in Figure 1, with the mean (SD) difference score between the percent correct as indicated by the two measures of 11.38 (±23.95) and 95% levels of agreement between −35.56 and 58.31. Lin’s concordance correlation coefficient was statistically significant (p <.001) suggesting a non-chance association, though this may be artificially optimistic given the large sample size (n=1518) and low concordance value (rho=0.14). Still, the correlation between the difference scores and the mean is exceptionally low (.001), suggesting that within the ranges observed here, there is no bias in association.

Figure 1.

The Bland and Altman levels of agreement between AMT and MMSE percent scores

Diagnostic accuracy of AMT:

We considered MMSE-Thai 2002 scores as a reference standard diagnosis of cognitive impairment. The sensitivity, specificity, positive, and negative predictive values of AMT at different cut-off points for determining cognitive impairment are shown in Table 3. At the cut point of 8, AMT had high specificity (90.3%) but low sensitivity (12.8%) and high positive predictive value (87.2%). The ROC is shown in Figure 2. The ROC showed the best cut-off score for the AMT is around 5; however, it is clear that the cut-point of 5 errs on the side of being too sensitive, but not specific enough to screen for cognitive impairment. While a larger value of area under ROC suggests the better overall performance of a diagnostic test,30 the AUC shown in Figure 2 and Table 3 are all small, ranging from .49 to .58.

Table 3.

The sensitivity, specificity, positive and negative predictive values of AMT at different cut-off points

AMT cut point	Sensitivity (%)	Specificity (%)	Positive Predictive Value (%)	Negative Predictive Value (%)	Area under ROC
0	100	0	83.7	16.3	.500
1	97.7	8.1	84.5	40.8	.529
2	96.3	10.5	84.7	35.6	.534
3	94.4	13.0	84.8	31.1	.537
4	91.7	18.2	85.2	30.0	.550
5	86.9	27.9	86.1	29.4	.574
6	77.6	38.9	86.7	25.2	.582
7	60.8	54.3	87.2	21.2	.575
8	12.8	90.3	87.2	16.8	.516
9	0.4	99.2	71.4	16.2	.498
10	0.2	100	100	16.3	.501

Figure 2.

The Receiver Operating Characteristic Curves (ROC) of AMT

Discussion

The following discussion is organized based on the objectives of this study and our results.

National cognitive screening tools

Thailand uses the AMT as an initial screening for cognitive dysfunction which is administered by volunteer health workers in villages. If the older adults scores <8 which indicates abnormal cognitive function, the volunteer health workers would send them to the sub-district health promotion hospital to be evaluated by a nurse using the MMSE-Thai.⁸ The AMT has been used for routine population screening for older adults in Thai community settings. There is concern regarding its use in such settings as the original version of AMT was designed to measure cognitive impairment, confusion state, or dementia of patients in hospitals.¹⁰ Several studies demonstrated that the AMT has high sensitivity and specificity in older adults compared with MMSE in populations other than Thailand.^31–33 The Spanish version of AMT showed a sensitivity of 100% and a specificity of 82.4% for a cut-off score of 7.¹³ A similar study in Hong Kong showed a sensitivity of 96% and a specificity of 94% for a cut-off score of 6.¹⁷ For cut-off scores of 6 and 7, a study in Iran showed the optimum balance between sensitivity (99% and 94%, respectively) and specificity (85% and 86%, respectively).¹⁴

However, our study found different results than those just described. At the cut-point of 8, AMT achieved high specificity (90.3%) but low sensitivity (12.8%) and high positive predictive value (87.2%). Our ROC showed that the best cut-point of 5 errs on the side of being highly sensitive but not specific enough to screen for cognitive impairment in this study. As only those older adults considered to be the most at-risk of cognitive decline by the AMT were given the MMSE test, it is not possible to compare the sensitivity, specificity, and the agreement of both measures for determining cognitive impairment in individuals under standard circumstances.

Methodology for determining the incidence of cognitive impairment in Thai community-dwelling older adults

The proportion of older adults identified as having cognitive impairment in Thai older adults as defined by the AMT (87.7%) was five times higher than indicated by the MMSE-Thai 2002 (16.3%). The result of cognitive impairment, as indicated by the AMT, could lead to anxiety, labeling, and stigmatizing of older adults who have a normal cognitive function. Since, in Thai society, the older adults usually remain important and highly respected persons in families, especially in rural areas, older adults might come to distrust and lack confidence in volunteer health workers’ performance in screening cognitive function.

The most poorly performed item in AMT was item 8, asking about ‘the year of great sorrow on October 14’ which basically measured remote memory. This situation happened in 1973 in Bangkok, the capital of Thailand.²⁵ The high rate of failure on this item suggests that it is not suitable for the entire Thai population, especially those who live in rural areas or with lower education levels. Adjustments to this item may be necessary based on older adults’ experiences as they relate to an important national occurrence or situation. We suggest further evaluation for validity using a more commonly understood national event in Thailand.

It is also interesting that almost half of older adults with cognitive impairment could not answer the items related to the current year and date of birth. This may be a result of the differing Thai cultural contexts. A study of age and birth date reporting in Thailand ³⁴ indicated that it was complicated to determine birthdate and calculate age because Thailand has two systems to report birthdates. A traditional practice among the rural population uses lunar months and a twelve-year cycle of animal years which coexists with a modern system stated in terms of western months and Buddhist Era (B.E.). Even on official birth registration forms, birth dates using both modern and traditional systems are recorded. Also, regional differences in familiarity and use of these methods may account for the results in our study. Another item that might be confusing for older adults at this time was item 9 asking about “the name of the present king?”. Due to the situation of King Rama IX’s being the longest-reigning monarch of Thailand (70 years) combined with his death in 2016, asking this question in 2017 might have been confusing for older adults. Moreover, there is a concern about content validity and whether the Thai AMT reflects all concepts within the AMT construct. In the original version, item 3 evaluated recent memory by asking “the address for recall” at the end of test.³¹ In the Thai version item 3 asks about the current address, meaning that this item did not measure the true theoretical meaning of the concept of recent memory as the original version.³¹

We conclude that the overall test validity of the AMT is in question because of the issues with the five items as described above. Also, in our sample, the area under the ROC curve, an indication of test accuracy, ranged from .49 to .58, considerably less than the optimum value.³⁵

For the MMSE, the proportion of older adults with cognitive decline indicated by the MMSE-Thai 2002 compares favorably with other global prevalence studies of cognitive decline, e.g., 4.4 – 17.1%. For example, in our sample, incorrect answers on the MMSE-Thai related to attention /calculation, with only 10.8% of the sample completing serial subtractions accurately. These results are similar to findings reported in most other countries using the test.^36–37

Limitations

Since we determined the incidence of cognitive impairment screening of older adults living in the community from the real situation in Thailand, there are some methodological limitations in this study. These include the retrospective design, the lack of gold standard from clinical diagnosis, the use of MMSE for comparison alone while the performance of the MMSE Thai 2002 may also be problematic in assessing cognitive impairment. Additionally, using the data from one province in the northeastern part of Thailand, might not generalize to the other regions of Thailand; however, our findings highlight the need to consider cultural and educational factors at the national level.

Conclusions:

As has been pointed out, the screening of cognitive impairment is a challenge in indigenous populations because of the educational and cultural inadequacy of the AMT in the Thai version. The item about “current address” did not measure the true theoretical meaning of the concept of recent memory as the original. Some items of the AMT need to be modified to improve the utility of this test. The cognitive screening of community-dwelling older adults in Thailand requires a standardized cross-cultural cognitive tool as well as a balance of benefits and harms resulting from the testing. Therefore, further study needs to occur to evaluate the overall validity of the AMT and its use as the screening measure for cognitive impairment in community-dwelling older adults in Thailand.

Implications for nursing practice.

When administered by volunteer health workers, the AMT may possess limited reliability and validity. Since nurses have an important role in screening for community awareness and preventive health, cognitive screening administered by nurses should be considered to help detect cognitive impairment older adults dwelling in the community.