The Memory Assessment Scale: A Population-based Cognitive Impairment Screening Instrument

Abstract

This study re-examined the Memory Assessment Scale (MAS), a brief memory test developed in Hawai‘i in 1987, to assess whether it remains a valid and reliable cognitive impairment screening tool in Hawai‘i. Patients suspected of having neurocognitive dysfunction were divided into 2 groups (those with and without mild cognitive impairment) based on their results on a battery of neuropsychological tests. No differences in MAS scores were found between patients with and without mild cognitive impairment. Further research with the MAS comparing patients with mild cognitive disorder to healthy controls is indicated to further examine the efficacy of this population-based test.

Introduction

The most widely applied screening instrument for the assessment of cognitive dysfunction is the Mini-Mental State Examination (MMSE),¹ but a meta-analysis concluded that MMSE has limited ability to differentiate between mild cognitive impairment and healthy controls.² Further, a systematic review of brief cognitive screening instruments recommended that clinicians should not consider 1 screening instrument, like the MMSE, be used in every setting.³ Instead, the reviewers cited the importance of population-based validation of screening tests, with data consisting of pertinent reference values that can serve to evaluate how well a person performs compared with a relevant population. The reviewers reported an unfortunate lack of instruments that are validated in a population-based cohort.

In Hawai‘i, where the multi-ethnic diversity in the population is particularly unique in the United States, the need for a relevant population-based cognitive screening instrument is especially important. According to the 2021 US Census Bureau American Community survey, Hawai‘i‘s population is approximately 37.5% Asian, 23.7% White, 10.6% Native Hawaiian and other Pacific Islander, 1.9% Black, 0.3% American Indian and Alaska Native, 1.6% “some other race,” and 24.4% “two or more races.”⁴ While the total population in Hawai‘i has increased significantly since the 1980s, from about 964 691 to 1 453 498 in 2021, the ethnicity mix is approximately the same.

The only known cognitive screening tool developed in Hawai‘i is the Memory Assessment Scale (MAS) that was introduced in a study in the 1980s.⁵ The results of the research that examined the efficacy of the MAS with neurological patients revealed an internal consistency reliability coefficient of 0.94 and an accuracy rate of 83% in identifying memory impairment in patients diagnosed with a neurocognitive disorder. The correlation between the MAS and the Luria Nebraska Neuropsychological Battery Intermediate Memory Scale was 0.63 (P<.01). The assessment of memory as with the MAS is critical because memory impairment is the first and most severely affected cognitive domain in dementia,⁶ and is generally considered the best predictor of cognitive decline.⁷

Since the original study, the MAS has not been investigated empirically. The purpose of the present research was to re-examine the efficacy of the MAS in Hawai‘i. With the growing population of the elderly, this research was driven by the desire to assess a brief memory test instrument that can identify individuals who are at risk for developing dementia, because memory deficits have been found to be early indicators of subsequent cognitive decline in older individuals. A review of over 40 brief memory tests developed to identify patients with mild cognitive impairment raised questions about the high risk of bias of many of the existing tests due to the unblinded evaluations comparing patients from memory clinics with diagnosed cognitive deficits to patients assumed to have no cognitive disorder.⁸ The present study avoided the bias of past studies that compared patients with known cognitive deficits versus healthy normals, by examining only participants with questionable neurocognitive dysfunction.

Methods

Participants

The study involved archival review of 114 patients (78 men, 36 women) who underwent comprehensive neuropsychological examination at Straub Medical Center due to questionable neurocognitive conditions. Many of the participants suffered mild traumatic head injury at work, in a motor vehicle accident, or at home. Other conditions included stroke, seizure, electrocution, near-drowning, depression, and IV epinephrine overdose. The average age of the patients was 44.1 years (SD = 15.8; Range 18–81). The average educational level was 14.0 years (SD = 2.5). The average duration between onset of injury and date of testing was 33.1 months (SD = 29.7).

To assess the efficacy of the MAS, 2 groups of patients were formed based on their performances on the Halstead-Reitan Neuropsychological Test Battery⁹: a Mild Cognitive Disorder group (n = 53) that consisted of patients who exhibited significant cognitive impairment in their test results, and a No Cognitive Disorder group (n = 61) that consisted of patients who had no cognitive impairment according to their test results. The MAS scores of the 2 patient groups were compared. The demographic characteristics of the participants are presented in Table 1. Data pertaining to the head injury conditions can be seen in Table 2. This archival study, with data acquired at Straub Clinic, was evaluated by the Hawai‘i Pacific Health Research Institute and was determined to be exempt from Institutional Review Board review.

Table 1.

Memory Assessment Scale Study Participant Demographic Information

	Mild Cognitive Disorder (n=53) Number (%)	No Cognitive Disorder (n=61) Number (%)	Test statistic	P-value
Sex
female	17 (47%)	19 (53%)	0.0113	92a
male	36 (46%)	42 (54%)
	Mean (SD)	Mean (SD)
Age (years)	44.4 (17.5)	43.9 (14.3)	0.105	.92b
Education (years)	14.1 (2.8)	13.9 (2.3)	−0.218	.83b
Time from injury to evaluation (months)	30.3 (28.2)	35.3 (30.9)	1.254	.21b
Positive neurodiagnostic test c	15 (28.3%)	15 (24.6%)

^aChi-square test

^bMann-Whitney U test

^cHalstead-Reitan Neuropsychological Test Battery

Table 2.

Types and Frequencies of Head Injuries and Illnesses

	N = 114
Motor Vehicle Accident	61
Fall	14
Other (head injury, head trauma on boat, head trauma by airplane turbulence, struck on head, hit by beam, by rock, by sledgehammer, by elevator door; electrocution, near drowning, intravenous epinephrine overdose)	11
Illness (dementia, stroke, seizure, multiple sclerosis, neurological disorder, AIDS, depression)	10
Assault/Fight	6
Head Injury at Home	7
Explosion (military, civilian)	3
Unknown	2

Measures

The MAS is a 33-item questionnaire comprised of items from standard memory scales and psychiatric interview schedules.⁵ See Table 3. It is administered by a technician and can be completed in less than 10 minutes. The MAS has been used as a simple introductory tool for the patients and was not considered in the interpretation of the neuropsychological test battery results. In addition to the MAS, the patients were also administered the Wechsler Memory Scale-3rd Edition (WMS-III),¹⁰ as part of a comprehensive neuropsychological examination. WMS-III is a widely used, individually administered measure of memory for immediate and delayed recall, and is a standard component of neuropsychological testing. From the WMS-III, 2 scores, the Immediate Memory Index (IMI) and the General Memory Index (GMI), were utilized in this study.

Table 3.

Memory Assessment Scale Instrument⁵

Statistical Analyses

The data were analyzed using parametric and non-parametric statistics to compare the 2 groups with regard to age, sex, education, and duration between onset of injury and date of examination. The reliability of the MAS was examined for inter-item consistency with the Kuder-Richardson Formula 20 test. Logistic regression evaluated the ability of MAS to predict Mild Cognitive Disorder patients when adjusting for IQ and memory scores. Spearman’s rank correlation coefficients were obtained to assess the relationship between the MAS with IMI and with GMI.

Receiver operating characteristic (ROC) curves and area under the curve (AUC) examined the diagnostic performance of the MAS, IMI, and GMI in identifying Mild Cognitive Disorder patients. Using varied cut-off scores for the MAS and known Cognitive Disorder status, the sensitivity and specificity levels of the MAS were determined. Lastly, univariate item analysis (Fisher’s Exact) was conducted for questions that were consistent across respondents (based on the previous Kuder-Richardson Formula 20 test) to identify MAS test items that were able to differentiate between Mild Cognitive Disorder and No Cognitive Disorder patients. Stata IC 15.0 software was used for all statistical analyses (StataCorp, College Station, TX). Findings were considered statistically significant at P<05.

Hypotheses

Based on the results of the prior study of the MAS, it was hypothesized that the MAS scores of patients in the Mild Cognitive Disorder group would be significantly lower than patients in the No Cognitive Disorder group. Significant correlations between the MAS with the IMI and with the GMI would be found.

Results

Table 4 presents the memory test scores of the Mild Cognitive Disorder and No Cognitive Disorder groups. Differences between the 2 groups in age, sex, education, and duration between injury and testing date were not statistically significant. The inter-item consistency results, with a Kuder-Richardson coefficient of 0.81, supported the homogeneity and reliability of the test items in the MAS. The logistic regression yielded an odds ratio of 0.88 (95% Cl [0.77, 1.01]), which indicated that the MAS score was not predictive of Mild Cognitive Disorder. The Spearman correlation coefficient between the MAS and the GMI was 0.36, and between MAS and the IMI was 0.32. The correlations were low but statistically significant (P<.001).

Table 4.

Memory Test Scores of Two Cognitive Disorder Groups

	Mild Cognitive Disorder (n=53) Number (%)	No Cognitive Disorder (n=61) Number (%)	Test statistic	P-value
Memory Test Scores
MAS	28.8 (4.4)	30.1 (2.3)	1.491	.14a
IMI	87.6 (17.9)	92.5 (16.7)	1.4677	.15b
GMI	87.5 (18.8)	95.8 (16.7)	2.4752	.01b

Tests: MAS=Memory Assessment Scaled, IMI=Immediate Memory Index, and GMI=General Memory Index

^aMann-Whitney U test

^bTwo-sample t-test

The ROC curves and AUC analyses indicated that MAS (0.58), IMI (0.58), and GMI (0.62) were not able to identify Mild Cognitive Disorder patients the majority of the time. In comparison, the GMI (0.62) was more likely to identify Mild Cognitive Disorder patients when compared to MAS and IMI. Figure 1 presents the ROC curves for the MAS, IMI, and GMI, compared to the diagonal line that a represents a random ability to identify Mild Cognitive Disorder patients.

Figure 1.

Receiver Operating Characteritic (ROC) Curves for Identification of Mild Cognitive Disorder by Neuropsychological Test

Using a cut-off score of 32, the sensitivity level of the MAS was 83% and its specificity was 26%. With a cut-off score of 31, the MAS sensitivity was 54.7%, while specificity was 50.8%. A cut-off score of 30 resulted in a MAS sensitivity of 39.6% and a specificity of 73.8%. Item analysis, using Fisher’s Exact Test, found 5 MAS items that significantly differentiated the Mild Cognitive Disorder and No Cognitive Disorder groups. They were questions No. 6, “What is your address?” (P = .04), No. 14, “What month is this?” (P = .02), No. 22, “Who is the President of the United States?” (P = .02) No. 30, “Did I ask you about the kind of work you do?” (P = .037), and No. 32, “What is my name?” (P = .038).

Discussion

The aim of this study was to re-examine the Hawai‘i-based MAS that was shown in 1987 to be effective in identifying patients diagnosed with a neurocognitive disorder.⁵ Unlike the previous MAS study in which the MAS scores of cognitively impaired patients were significantly lower than the patients without cognitive impairment, the present investigation found no significant difference in MAS scores between the Mild Cognitive Disorder and No Cognitive Disorder patients. An apparent reason for the disparate findings was that the cognitively impaired participants in the earlier study were comprised of more severely impaired patients, all of whom evidenced abnormal findings in a neurological examination as well as in a neurodiagnostic test, as compared to the patients without cognitive impairment who had negative neurological examinations and neurodiagnostic test results. In the present study, only 28.3% of the Mild Cognitive Disorder patients and 24.6% of the No Cognitive Disorder patients had a positive neurodiagnostic test result. In short, the neurological conditions of the 2 current groups of participants in this research may not have been substantially different from each other, thus obtaining relatively similar MAS memory scores.

If the purpose of the screening test is to identify individuals who may have a cognitive disorder, to be followed by a safe and more specific test for definitive diagnosis, or by a low-cost, low-risk intervention, a lower MAS cut score that increases diagnostic sensitivity but with a corresponding decrease in specificity may be desirable.¹¹ In that case, an MAS cut-score of 32 that results in a sensitivity of 83% and specificity of 26% would be preferable. On the other hand, if the objective is to identify patients with a brain disorder who would then need a costly or invasive next step, such as positron emission tomography scanning or lumbar puncture, it may be better to maximize specificity to minimize unnecessary major procedures in patients incorrectly classified as having cognitive disorder. In that situation, an MAS cut-off score of 30 would result in a sensitivity level of 39.6% and specificity of 73.8%. Ultimately, the clinician using the MAS needs to select the cut-scores optimized for their purpose. Users of screening tests should strike a balance between sensitivity and specificity to rule in or out the participants with cognitive disorder.¹²

In this study, the diagnostic performance of MAS was limited in identifying Mild Cognitive Disorder patients, with an AUC of 58%. Reviewers of screening tests for cognitive impairment concluded that brief instruments, like the MMSE or MAS, suffer from their brevity and limited coverage of abilities. Thus, while a brief 10-minute test fits well within a time-constrained medical visit, a comprehensive assessment of skills would require 4 to 6 hours of administration time to retain stronger psychometric qualities.¹³ Additional disadvantages of the more informative comprehensive cognitive testing is that it requires a battery of testing equipment and trained staff that are not compatible with a standard clinical visit, as well as entailing further costs.

The value of a population-based psychometric instrument, like the MAS, cannot be overstated. Population-based tests that are validated with persons who closely resemble the group to which an individual belongs provide the best comparisons that maximize the accurate diagnosis of brain-impaired persons.¹⁴ With revisions, the MAS, as a population-based memory scale, has the potential to be a valuable asset for clinicians in Hawai‘i seeking a screening test to assist in the detection of patients with mild memory impairment. The data from this study can be used to improve the accuracy of the MAS with item analyses, preserving test items that are effective in differentiating those with and without cognitive disorder, and to conduct a follow-up study with individuals 60 years and older.

Limitations

Several limitations ofthis research are noted. The relatively small sample sizes of the 2 groups could have lowered the statistical power of the analyses, contributing to the null findings. The smaller sample limited the ability to reveal differences, and this is particularly relevant in this research because the majority of Mild Cognitive Disorder patients had mild neuropsychological impairment. A comparison with a larger sample size could have enhanced the diagnostic capacity of the MAS in identifying memory deficits in mild cognitive impairment. This retroactive study did not systematically require neurodiagnostic tests that would be valuable in understanding those with memory difficulties. The reliability of the MAS was evaluated with an inter-item consistency measure, whereas a preferred test-retest reliability measure was not possible with a single test administration. Additional MAS research with a larger sample of patients exhibiting varied severity of cognitive dysfunction should be considered to further assess the efficacy of this screening test in Hawai‘i. A study that compares the MAS scores of patients with mild cognitive disorder with healthy controls would be especially beneficial.

Conclusions

The present study found a significant correlation between the MAS and the WMS-III, a widely used memory test. However, no differences in MAS scores were found between patients with and without mild cognitive disorder. Revision of the MAS may be needed for it to be utilized as a population-based cognitive screening instrument.

Glossary

Abbreviations

AUC

area under the curve

GMI

General Memory Index

IMI

Immediate Memory Index

MAS

Memory Assessment Scale

MMSE

Mini-Mental State Examination

ROC

receiver operating characteristic

WMS III

Wechsler Memory Scale III

Conflict of Interest

None of the authors identify a conflict of interest.