Sensitivity and Specificity of the Mini-Mental State Examination for Identifying Dementia in the Oldest-Old: The 90+ Study

Abstract

OBJECTIVES

To evaluate the sensitivity and specificity of the Mini-Mental State Examination (MMSE) in identifying dementia in the oldest-old when stratified by age and education.

DESIGN

Cross-sectional.

SETTING

Research clinic and in-home visits.

PARTICIPANTS

Population-based sample of adults aged 90 and older (n = 435) who are enrolled in the 90+ Study, a longitudinal, population-based study.

MEASUREMENTS

Neurological examination to determine dementia diagnosis, MMSE, and demographic data.

RESULTS

Receiver operating characteristic (ROC) analyses indicated that the MMSE had high diagnostic accuracy for identifying dementia in subjects aged 90 and older across different age and education groups (area under the ROC curve values ranged from 0.82 to 0.98). A range of possible cutoff values and corresponding sensitivity and specificity are provided for the following age groups: 90–93, 94–96, and ≥97. Age groups were subdivided by educational attainment (≤high school, vocational school or some college, college degree or higher). In subjects aged 90 to 93 with a college degree or higher, the suggested MMSE cutoff score is ≥25 (sensitivity = 0.82, specificity = 0.80). In those aged 94 to 96 with a college degree or higher, the suggested cutoff is ≤24 (sensitivity = 0.85, specificity = 0.80). Those aged 97 and older with an education of high school or less had the lowest suggested cutoff ≤22 (sensitivity = 0.80, specificity = 0.76).

CONCLUSION

Overall, the MMSE had good sensitivity and specificity across all age and educational groups. Optimal cutoff points were lower in the older age groups and those with less education, primarily to preserve specificity. This screening instrument is appropriate for use with the oldest-old.

The age composition of the United States is rapidly changing. An increasing proportion of the U.S. population is aged 65 and older, and advances in public health and medicine contribute to an extended lifespan. In 2000, approximately 1.4 million Americans were aged 90 or older, and that number is expected to reach nearly 10 million by 2050.^1,2 With the relatively new emergence of this age group, little is known about the psychological, medical, and social aspects of living into one's tenth decade of life. Studies of the oldest-old are needed to better characterize specific adaptations in healthcare practices required by this unique group.

For lay persons and healthcare providers, cognitive decline often is thought of as an unavoidable part of the aging process. This expected link between aging and deteriorating cognition is likely due in part to age-related increases in incident dementia beginning at approximately the age of 65.^3,4 Although age is a significant risk factor for dementia, not all older adults develop a dementing illness. Differentiating dementia from normal age-related changes in cognition is a challenge for researchers and healthcare providers working with older people. Several valid and reliable dementia screening instruments have been developed to address this, including the Blessed Dementia Scale,⁵ Dementia Rating Scale,⁶ Mini-Mental State Examination (MMSE),⁷ and Modified Mini-Mental State Examination (3MS)—an expanded version of the MMSE.⁸ These screening instruments are essential tools in identifying possible cognitive impairment, as well as monitoring decline in global cognitive functioning.

Of the many different screening tests used in practice, the MMSE is probably the most widely used instrument, owing in part to its ease of administration, sampling of multiple cognitive domains, and validity in detecting dementia.^9,10 Although the psychometric properties of the MMSE for various populations are well documented in the literature, little information is available on the sensitivity and specificity of this screening instrument in the oldest-old. The proper use of the MMSE with nonagenarians and centenarians depends on the availability of appropriate cutoff scores that take into account age and education. This study presents the sensitivity and specificity of the MMSE for identifying dementia in a sample of subjects aged 90 and older. These data are presented to help researchers and healthcare providers appropriately interpret this screening instrument when applied to their oldest-old patients. The sensitivity and specificity are analyzed according to age and education subgroups. Suggested cutoff points are provided that best balance sensitivity and specificity.

METHODS

Participants were drawn from the 90+ Study, a southern California–based epidemiological investigation of longevity and brain aging in persons aged 90 and older. These subjects are from a cohort that was first contacted in the early 1980s as part of the Leisure World Cohort Study, an epidemiological investigation of a large retirement community. Beginning in 2001, the 1,150 participants from the original cohort who were aged 90 and older were invited to join the 90+ Study. As of April 2006, trained neurological examiners and neuropsychological testers who collected data on medical history, medication usage, functional ability, social engagement, neuropsychological functioning, and neurological status recruited and examined more than 500 people in person. The institutional review board of the University of California, Irvine, approved this research, and informed consent was obtained from all participants or their legal representatives.

Participants in these analyses were limited to those completing all items on the MMSE. The 30-point MMSE is embedded within the 3MS, and a derived MMSE score is computed by adding its 11 items. Because there are additional items on the 3MS, the order of presentation for MMSE items deviates from the standard MMSE administration and presents items in order of registration, mental reversal, recall, temporal orientation, spatial orientation, naming, repetition, read and obey, writing, three-stage command, and pentagon copy. The three registration and recall words used were “brown,” “shirt,” and “honesty.” The mental reversal item was “world” spelled backward rather than serial 7 s. A more-detailed description of the neuropsychological battery and its administration is available in a normative study of the 90+ Study participants.¹¹

Every 6 months, trained neurological examiners who assessed the participants in person assign participants in the 90+ Study a diagnosis according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria for dementia. The neurological examiners observed the administration of the 3MS, from which the MMSE was derived, to supplement their own cognitive status assessment but were not made aware of the total score. Participants were classified into one of three diagnostic categories: normal, cognitively impaired—not demented (CIND), and demented. The normal group includes participants with no cognitive or functional loss. The CIND group includes participants with functional loss and only one domain of cognitive decline or with no functional loss and two or more domains of cognitive decline, thus not meeting criteria for dementia. For purposes of the analyses, participants with normal cognition and CIND were combined in the nondemented group. The demented group included all participants meeting DSM-IV criteria for dementia.

Participants were stratified into age and education categories that resulted in the most even distribution of participants. The age categories were 90 to 93, 94 to 96, and 97 and older. Age groups were subdivided by education attainment as follows: high school or less (≤HS), vocational school or some college, and college degree or higher (college+).

Sensitivity and specificity were calculated using a receiver operating characteristic (ROC) curve that plots sensitivity and specificity across the range of possible cutoff scores. ROC analyses give an estimate of the overall discriminative ability of a scale using the area under the curve (AUC) statistic. The AUC provides an indication of how close a scale is to the ideal point of 100% sensitivity and 100% specificity, and it is not dependent on any particular cutoff. Sensitivity is defined as the proportion of true positives over combined true positives and false negatives in the entire sample. Specificity is the proportion of true negatives over combined true negatives and false positives in the entire sample.

All analyses were performed using SPSS, version 13.0, statistical software for Windows (SPSS Inc., Chicago, IL) and SAS, version 7.1 (SAS Institute, Inc., Cary, NC).

RESULTS

Of the 524 90+ Study participants who were seen in person for evaluation, 435 met inclusion criteria with completion of all items on the MMSE, complete demographic information, and having a diagnosis assigned by a neurological examiner. Demographic characteristics of the sample are presented in Table 1. The mean age of this sample was 95 (range 90–104). The sample was 74% female, and 73% of the participants had more than 12 years of formal education. Based on the neurological examiner's diagnosis, 36% of participants were normal, 28% CIND, and 36% demented. Eight-six participants did not meet inclusion criteria because of failure to complete all items on the MMSE due to severe dementia, fatigue, physical impairment, vision impairment, hearing impairment, or refusal of testing. Three additional participants did not report their educational attainment and were excluded. The participants who were not included in the analyses were older (mean age 96.01 ± 3.62 vs 94.98 ± 3.02, P = .01), had proportionally less education (71% of noncompleters vs 73% of completers with >high school, P=5.67), and had a greater proportion with a diagnosis of dementia (71% of noncompleters vs 36% of completers diagnosed with dementia, P<.001).

Table 1.

Demographic and Descriptive Statistics of Age-by-Education Subgroups

Age-by-Education Subgroup	Subgroup Size n	Female %	Dementia %	Mini-Mental State Examination Score Mean (Out of 30)	Area Under the Receiver Operating Characteristic Curve
Aged 90–93
≤HS	49	74	31	23	.92
Vocational school or some college	57	81	21	24	.98
College+	63	68	35	23	.90
Aged 94–96
≤HS	35	89	57	19	.94
Vocational school or some college	43	77	40	23	.94
College+	74	60	27	23	.92
Aged ≥97
≤HS	32	84	47	19	.82
Vocational school or some college	33	85	49	22	.93
College+	49	67	37	21	.95

HS=high, school; College+=college education or more.

Table 1 presents the number of participants included in each age according to education subgroup and mean MMSE and AUC values for age and education subgroups. The mean MMSE scores in the group aged 90 to 93 were approximately the same for all levels of educational attainment. In the groups aged 94 to 96 and 97 and older, the ≤HS group had the lowest mean MMSE scores. The lowest AUC value (0.82) occurred in the group aged 97 and older with the least education. The highest AUC value (0.98) was for group aged 90 to 93 with vocational school or some college. Figure 1 shows the ROC curves representing the lowest and highest AUC values, as well as the ROC curve for the subgroup aged 94 to 96 with more than a college education, whose AUC value of 0.92 is the same as the overall mean AUC value of 0.92. All other AUC values for the MMSE ranged from 0.90 to 0.95, indicating excellent overall accuracy at identifying dementia across age and education stratifications.

Figure 1.

Selected receiver operating characteristic (ROC) curves. ROC curves represent the overall discriminability of the Mini-Mental State Examination for identifying dementia in specified age×education subgroups. Subgroup sizes, sex distribution, diagnostic distribution, mean Mini-Mental State Examination, and area under the curve (AUC) values for age-by-education subgroups.

The optimal cutoffs were determined by finding the MMSE values that allowed for the best balance between sensitivity and specificity. A range of possible cutoff values is provided in Table 2. For the MMSE, the optimal cutoff for dementia in those aged 90 to 93 was highest in the group with college or more education (≤HS cutoff ≤23, vocational or some college cutoff ≤22, college+cutoff ≤25). For the group aged 94 to 96, the optimal cutoffs by educational attainment were ≤23 for ≤HS, ≤25 for vocational or some college, and ≤24 for college+. The group aged 97 and older had the following optimal cutoffs by education: ≤22 for ≤HS, ≤24 for vocational or some college, and ≤22 for college+.

Table 2.

Sensitivity and Specificity of Mini-Mental State Examination (MMSE)

		Education
		High School or Less		Vocational School or Some College		College Graduate or Graduate School
Age	MMSE Cutoff	Sensitivity	Specificity	Sensitivity	Specificity	Sensitivity	Specificity
90–93	21	.80	1.0	.83	.96	.73	.98
	22	.80	.97	.92*	.96*	.77	.95
	23	.87*	.94*	.92	.93	—	—
	24	.87	.79	—	—	.77	.88
	25	.87	.65	1.0	.80	.82*	.80*
	26	.87	.59	1.0	.67	.86	.73
	27	.93	.35	1.0	.49	.96	.54
	28	1.0	.26	1.0	.29	.96	.41
94–96	21	.85	.93	.59	.96	.60	.96
	22	—	—	.65	.92	.75	.96
	23	.90*	.93*	.71	.88	.80	.94
	24	.90	.87	.71	.85	.85*	.80*
	25	.95	.87	.88*	.85*	.90	.74
	26	.95	.53	1.0	.77	.95	.68
	27	1.0	.47	1.0	.58	.95	.52
	28	1.0	.20	1.0	.23	.95	.33
≥97	21	.73	.82	.69	1.0	.83	.90
	22	.80*	.76*	.81	.88	.89*	.90*
	23	.80	.71	.88	.88	.94	.81
	24	.80	.59	.94*	.88*	.94	.58
	25	.87	.35	.94	.82	1.0	.58
	26	.93	.18	—	—	1.0	.52
	27	.93	.06	.94	.65	1.0	.42
	28	1.0	0.0	.94	.35	1.0	.23

Note: Sensitivity and specificity of individual cutoff scores for age-by-education subgroups.

^*Best combination of sensitivity and specificity for each subgroup.

—=no participant in that cell obtained the specific MMSE value.

DISCUSSION

The purpose of this study was to provide cutoff scores on the MMSE for the detection of dementia in subjects aged 90 and older stratified according to age and education. The sensitivity and specificity, as well as the overall accuracy, of the MMSE at detecting dementia in the oldest-old were assessed. The current results support making adjustments for age and education when determining cutoff scores for dementia on the MMSE in a sample of people aged 90 and older.

Across all age and education groups, the MMSE demonstrated good overall diagnostic accuracy, as reflected by high AUC values (Table 1 and Figure 1). The AUC values ranged from a low of 0.82 in the group aged 97 and older with a high school education or less to a high of 0.98 in the group aged 90 to 93 with vocational or some college education. In the group with the lowest AUC, the specificity values were notably lower than in other groups. In those aged 97 and older with a high school education or less, the MMSE is a less-accurate screening tool than for other age and education groups. This may be due in part to a greater proportion of people aged 97 and older with a high school education or less that have some cognitive impairment and were included in the nondemented group. Across all education groups in those aged 97 and older, more participants were CIND than were normal (32% CIND, 25% normal), but in those aged 90 to 93, fewer participants were CIND than were normal (28% CIND, 42% normal). The participants with cognitive impairment of insufficient severity to meet criteria for dementia likely had lower MMSE scores that resulted in more false positives, thus lowering specificity. In an extension of the current study, analyses examining the diagnostic accuracy of the MMSE for identifying CIND in the oldest-old are presently underway with participants of the 90+ Study.

One consistent finding in the scientific literature is that greater age is associated with lower MMSE scores.^11–15 The decline in MMSE scores across the later decades of life could be due to a decline in cognition related to benign age-related changes or, alternatively, may result from a higher proportion of cognitively impaired or demented persons in the oldest-old group. Regardless of the cause of age-related changes in MMSE, failure to adjust cutoffs based on age typically results in a loss of specificity.^16,17 Given the reported age effects, the cutoff scores for dementia on the MMSE need to be adjusted according to age to maximize the sensitivity and specificity of the scale.¹⁸

With advancing age, optimal cutoff scores were expected to decline regardless of educational attainment. Although the optimal cutoffs in this sample did not follow a direct linear trend of lower cutoffs with increasing age across all educational groups, the lowest optimal cutoff score was in the oldest age group. For instance, in the college+group, the recommended cutoff for dementia in those aged 90 to 93 was 25, decreasing to 24 for those aged 94 to 96, and further decreasing to 22 for those aged 97 and older. These downward adjustments were in favor of preserving specificity. Keeping the same cutoff of 25 for those aged 94 to 96 and 97 and older in the college+group would have lowered the specificity from 0.80 to 0.74 and 0.58, respectively. The decline in specificity with increasing age was similarly noted in findings from the Canadian Study on Health and Aging (CSHA).¹⁷ When using a cutoff of 24 in the CSHA sample, sensitivity was higher and specificity was lower in those aged 80 to 89 (sensitivity=0.95, specificity=0.82) than in those aged 65 to 79 (sensitivity=0.82, specificity=0.86). Age-adjusted cutoffs for the MMSE may help preserve specificity, thus allowing for accurate screening of dementia for elderly patients even into their 90s.

The unexpected finding of lower optimal cutoffs for younger participants found in the group with vocational school or some college education (22 for aged 90–93, 25 for aged 94–96, and 24 for aged ≥97) may be due in part to the number of specific MMSE scores represented in each subgroup. Subjects aged 94 to 96 had a more evenly distributed number cutoff scores (e.g., 2 participants with a score of 23, 1 participants with a score of 24, and 3 participants with a score of 25), whereas those aged 90 to 93 had one participant with a score of 23, no participant with a score of 24, and seven participants with a score of 25. Given the small sample sizes in the subgroups, these variations in represented scores possibly influenced the sensitivity and specificity estimations. Regardless of chosen cutoff scores, the group aged 90 to 93 had higher specificity than those aged 94 to 96 and 97 and older in the group with vocational school or some college education, in keeping with the trend of the other age-by-education groups in this sample. Therefore, age-adjusted cutoffs are likely to be useful when using the MMSE with subjects aged 90 and older

Along with older age, lower educational level is associated with lower MMSE scores.^13,15,19,20 Education-adjusted cutoff scores generally are thought to reduce the risk of misclassification bias in screening for dementia.²¹ Adjusting the MMSE cutoff for dementia to a lower cutoff score in patients with less education would help preserve specificity.

Results from the present study did not show a clear advantage for lowering the cutoff point for participants with less education. In the group aged 90 to 93, for example, the best balance between sensitivity and specificity occurred at a cutoff of 23 for participants with a high school education or less, decreased to a cutoff of 22 for those with vocational training or some college, and increased to a cutoff of 25 for those with a college education or higher. Furthermore, in the group aged 97 and older, the cutoff for the ≤HS and college+groups was the same when considering the best sensitivity versus specificity trade-off, whereas the optimal cutoff for the vocational school or some college group was higher. The unexpected variation in cutoffs for the vocational school or some college educational group in relation to the ≤HS and college+groups possibly reflects the heterogeneity of the vocational school or some college group. It is possible that those who attended vocational school are different from those who attended some college in a manner that would affect cognitive test performance, but the present data set does not allow a separation of the two groups for further analyses. Results from this sample suggest that education-adjusted cutoffs in those aged 90 and older may not be as useful as age-adjusted cutoff scores because of the variation seen in the vocational school or some college group. Furthermore, the small number of participants in this sample having a high school education or less precluded further analyses across a lower range of educational achievement.

Several caveats must be considered when evaluating these study results. First, the neurology examiners observed the administration of the MMSE items during participants' evaluations. As such, a participant's performance on the MMSE may have influenced the dementia diagnosis. This is unlikely to have significantly affected the outcomes, because the final MMSE score was not shared with the neurological examiner until after the entire protocol was completed and a diagnosis had already been assigned. Furthermore, the MMSE data were used within the context of a much longer neurological examination that included additional mental status assessment. Finally, no predetermined MMSE cutoff scores were used in the neurologist's diagnosis of dementia.

The second caveat relates to the generalizability of these findings for persons with low educational attainment. This sample represents a highly educated group of the oldest-old. Ninety-one percent of participants had completed at least high school, compared with approximately 53% of the general U.S. population in the same age cohort.¹ Optimal cutoffs for the oldest-old with a grade school education may be lower than those demonstrated with this sample.

This study has several strengths. It included a large number of subjects aged 90 and older, ranging from 90 to 104. This allowed a more-detailed analysis of age-related cutoff points than is currently available in the literature. In addition, the availability of a neurological diagnosis, including CIND, allowed reasonable inferences to be made as to the effect of a higher proportion of subjects aged 97 and older with a CIND diagnosis potentially affecting the specificity of the MMSE as a screening tool. Also, by presenting MMSE cutoff scores for dementia across the tenth decade, this study expands the usefulness of a tool that is already in widespread use in research and clinical practice.

The oldest-old are the fastest growing segment of the population, and having age-appropriate cutoff scores for screening measures such as the MMSE provides useful tools for the accurate screening and further treatment of cognitive dysfunction in these patients. The MMSE appears to be an accurate instrument for dementia screening in subjects aged 90 and older. To preserve a balance between sensitivity and specificity, suggested cutoff values on the scale were adjusted downward for older subjects and those with less education. Based on findings from the present study, age- and education-related adjustments should be considered when using the MMSE with patients who are aged 90 and older.