Using the Folstein Mini Mental State Exam (MMSE) to explore methodological issues in cognitive aging research

Abstract

Cognitive scales are used frequently in geriatric research and practice. These instruments are constructed with underlying assumptions that are a part of their validation process. A common measurement scale used in older adults is the Folstein Mini Mental State Exam (MMSE). The MMSE was designed to screen for cognitive impairment and is used often in geriatric research. This paper has three aims. Aim one was to explore four potential threats to validity in the use of the MMSE: (1) administering the exam without meeting the underlying assumptions, (2) not reporting that the underlying assumptions were assessed prior to test administration, (3) use of variable and inconsistent cut-off scores for the determination of presence of cognitive impairment, and (4) failure to adjust the scores based on the demographic characteristics of the tested subject. Aim two was to conduct a literature search to determine if the assumptions of (1) education level assessment, (2) sensory assessment, and (3) language fluency were being met and clearly reported in published research using the MMSE. Aim three was to provide recommendations to minimalize threats to validity in research studies that use cognitive scales, such as the MMSE. We found inconsistencies in published work in reporting whether or not subjects meet the assumptions that underlie a reliable and valid MMSE score. These inconsistencies can pose threats to the reliability of exam results. Fourteen of the 50 studies reviewed reported inclusion of all three of these assumptions. Inconsistencies in reporting the inclusion of the underlying assumptions for a reliable score could mean that subjects were not appropriate to be tested by use of the MMSE or that an appropriate test administration of the MMSE was not clearly reported. Thus, the research literature could have threats to both validity and reliability based on misuse of or improper reported use of the MMSE. Six recommendations are provided to minimalize these threats in future research.

Mrs. Gonzales a fictional vignette

Mrs. Gonzales is an 86-year-old living in an assisted-living facility affiliated with a large urban research-intensive university in the southwestern United States. Mrs. Gonzales is well liked by the staff, and though her primary language is Spanish, she has learned some conversational English. Overall, she has a positive outlook and has a relatively high degree of activity including going for walks, watching TV, playing cards, and taking Tai Chi classes. Because she was unable to continue school after the 6th grade, Mrs. Gonzales has recently been taking classes to improve her English and reading skills. Mrs. Gonzales’ demeanor and location of residence place her in a position to be included in geriatric research studies.

Two weeks ago, a research assistant was in the facility screening individuals to participate in a study about cognitively impaired older adults. The research assistant spoke and administered the Folstein Mini Mental State Exam (MMSE) in English. The principle investigators of the study wanted to include older adults with mild, moderate, and severe cognitive impairment in the study; therefore, inclusion criteria were nursing home residents with an MMSE score of 22 or less. Mrs. Gonzales scored an 18 on the exam administered by the research assistant and was enrolled in the study. Mrs. Gonzales was not wearing her hearing aid or glasses when the exam was administered.

Aims

This paper has three aims. First, we show four potential threats to validity in the use of the MMSE (1) by administering the exam without meeting the underlying assumptions, (2) by not reporting that the underlying assumptions were assessed prior to test administration, (3) by variable and inconsistent cut-off scores for the determination of presence of cognitive impairment, and (4) failure to adjust the scores based on the demographic characteristics of the tested subject. Second, we conducted a literature search to determine if the assumptions of (1) education level assessment, (2) sensory assessment, and (3) language fluency were being met and clearly reported in published research using the MMSE. Last, we conclude with recommendations to decrease the threats to validity that these challenges pose to research.

Introduction

Background to the MMSE

Dementia is general term describing a set of symptoms that lead to a decline in cognitive function. The most common symptom of dementia is memory loss and the most common type of dementia is Alzheimer’s disease (2008). Cognitive scales are used frequently in geriatric research and clinical practice to determine the presence of probable dementia. These scales are built on a set of underlying assumptions and these assumptions must be met when using these scales in order to obtain reliable and valid results. One of the most commonly used scales for screening cognitive ability is the MMSE. The MMSE was developed to be a brief, clinical, quantitative measure of cognitive status in adults (Folstein et al. 1975). The MMSE can be used to screen for cognitive impairment, to estimate the severity of cognitive impairment at a given point in time, to follow the course of cognitive changes in an individual over time, and to document an individual’s response to treatment (Folstein et al. 1975). The MMSE measures several subsets of cognitive status including comprehension, reading, writing, orientation, and drawing abilities (Folstein et al. 1975; Guerrero-Berroa et al. 2009). However, the divergent validity of the MMSE to accurately measure these subsets is questionable (Tierney et al. 1997). The MMSE is commonly reported as the instrument used to screen for cognitive impairment because it is simple to use and can be administered in less than 10 min. Since the introduction of the MMSE, some criticisms have appeared including (1) its failure to discriminate between people without dementia and those with mild dementia, (2) limited ability in detecting right hemisphere impairment, (3) simple language items that reduce sensitivity to mild language deficits, and (4) numbers of false-positives (Tombaugh et al. 1996).

The MMSE was created with a set of inherent underlying assumptions (Folstein et al. 1975). One assumption is that the subject being tested can read and write in the language of the administered exam. Subsequent research has demonstrated that the MMSE is biased toward people with less than an 8th grade level of education as they are more likely to falsely test positive for dementia (Tombaugh and McIntyre 1992) or mild cognitive impairment (Crum et al. 1993). A second assumption is that the subject must be fluent in the spoken language of the exam administration (Escobar et al. 1986; Mathuranath et al. 2004; Tiwari et al. 2009) in order to follow spoken instructions and to answer spoken questions. The third assumption is that the subject must possess the abilities to hear, talk, and see; these abilities are required to complete the exam (Kurlowicz and Wallace 1999; Foreman et al. 1996). Thus, people with sensory or language disorders may perform poorly on the MMSE yet they may or may not have cognitive impairment. For example, people with aphasia often score low (Chow et al. 2006) or they are screened out of studies (D’Alessandro et al. 1996). Although generally aware of these assumptions, researchers may not report whether these assumptions are met when discussing methods using the MMSE. This is a potential threat to validity and can make the results of studies open to question.

MMSE development

Folstein et al. (1975) developed the MMSE as a simplified form of the cognitive mental status examination. Their goal was to develop an instrument that could be quickly administered by clinicians to people who might only cooperate for short periods of time. Folstein et al. (1975) established the validity of the MMSE in two different studies by administering the exam to subjects with dementia, depression with cognitive impairment, and affective disorder, depressive type. In the first study, 59 subjects with either dementia or depression with cognitive impairment were compared to 63 “normal” subjects. The results showed that scores for subjects with dementia were significantly different from scores for the normal subjects.

In the second validity study (Folstein et al. 1975), 137 consecutive admissions to a psychiatric hospital were evaluated and again subjects with dementia were found to have significantly lower scores than subjects with depression with affective disorder, mania, schizophrenia, or personality disorder with drug abuse, and neuroses. Folstein et al. (1975) concluded that the MMSE was a valid measure of cognitive status. The reliability of the MMSE was measured by using 24-h and 28-day retest with single or multiple users. The correlation was r = 0.88 when given by the same testers over 24 h; the correlation using different testers was r = 0.82. The 28-day retest with a different set of subjects was r = 0.98.

Potential threats to validity

Appropriate use of the MMSE

The MMSE was designed for use as a screening device for cognitive impairment (Folstein et al. 1975; Tombaugh and McIntyre 1992). Positive screening means that cognitive impairment likely exists. The MMSE was not developed to diagnose dementia and should not be reported as doing such. The MMSE does not differentiate among the various causes of dementia such as Alzheimer’s disease or vascular dementia (Foreman et al. 1996). Accurate differential diagnosis of dementia requires additional neuropsychiatric testing (Reisberg 2007).

As previously mentioned the MMSE is not reliable in persons who cannot read or write or who are not fluent in the spoken language of the administered exam (Tombaugh and McIntyre 1992; Tiwari et al. 2009). Any sensory deprivation may bias the results of the examination (Foreman et al. 1996). The MMSE is available in several languages (Mystakidou et al. 2007), though item translation into other languages may bias scores (Ramirez et al. 2006).

Age and education considerations

Folstein et al. (1975) reported no differences in cognitive impairment between subjects under age 60 compared to those over age 60. Subsequent investigators concluded that age-related changes begin at age 55–60 and that age-related declines in MMSE scores increase more quickly in persons over the ages of 75 (Dufouil et al. 2000). Esther (2004) showed a consistent decline in MMSE scores from baseline over time in a study of 331 older adults (mean age 86.1, range 77–101). A reason for these declines may be undiagnosed neurodegenerative dementias and sensory deficits. Suggestions for age and education adjustment of cognitive scales are mixed. Some argue that adjusting for education and age are necessary (see Kittner et al. (1986) for review), while others contend that adjusting cut-off scores for age and education fails to identify people in the early stages of illness (Morris et al. 2010).

MMSE cut-off score considerations

Several differences exist in reported cut-off scores used in research studies (Wood et al. 2006) and this potentially causes confusion when comparing study results (Table 1). Changing the cut-off points alters both the sensitivity and specificity of the MMSE, increasing one while decreasing the other (Tombaugh and McIntyre 1992). Cut-off points are the specific scores on the instrument where the researcher or clinician determines mild, moderate, or severe cognitive impairment. Variations in these points can lead to differences in findings (Cullen et al. 2005). The original work of Folstein et al. (1975) reported that the MMSE should be scored from 0 to 30, with a score of 24 or greater as “normal” and with a score less than 20 “likely dementia.” Tombaugh and McIntyre (1992) provided an exhaustive review of the MMSE. They proposed the following scale: no impairment, 24–30; mild cognitive impairment, 18–23; and severe cognitive impairment 0–17.

Table 1.

Examples of inconsistent MMSE cut-off scores reported in the literature

Study	MMSE score	Descriptor used in study
Folstein et al. (1975)	20 or less 24–30	Likely dementia Normal
Tombaugh and McIntyre (1992)	24–30 18–23 0–17	No impairment Mild cognitive impairment Severe cognitive impairment
Chopra et al. (2007)	27–30 24–26 18–23 14–17 0–13	Normal Borderline impairment Mild impairment Moderate impairment Severe impairment
Ferrell et al. (2000)	19 or less	Cognitive impairment
Kaasalainen et al. (1998)	23 or less	Cognitive impairment
Scherder and Bouma (2000)	18	Serious cognitive disturbance
Krulewitch et al. (2000)	24	Mild cognitive impairment
Tsai et al. (2008)	9 or less 25–30	Severe cognitive impairment No cognitive impairment
Radbruch et al. (2000)	20/21	Impaired
Chopra et al. (2008)	14 or less	Severe cognitive impairment
Shega et al. (2008)	24 or greater 9 or less	Cognitively intact Severe cognitive impairment

Cut-off scores should be considered carefully. For example, we found several cut-off scores reported for severe cognitive impairment (Table 1). MMSE scores of 17 or less are generally reported as severe cognitive impairment (Kaasalainen 2007; Chopra et al. 2007, 2008; Shega et al. 2008). Yet, varying cognitive abilities may occur within the population of people who score 17 or less. Differences exist between people with moderate, moderately severe, severe, and very severe cognitive impairment (Paquay et al. 2007; Morris et al. 1994). MMSE scores of 15, 7, 5, and 1 have been shown to be associated with moderate, moderately severe, severe, and very severe cognitive impairment, respectively (Morris et al. 1994; Hartmaier et al. 1995). These variations in scoring may create challenges when comparing findings from different studies. Others have comprehensively reviewed cut-off scores (Crum et al. 1993; Tombaugh and McIntyre 1992; Dufouil et al. 2000; Cullen et al. 2005).

Other scoring considerations

Other considerations that may bias MMSE scores should be carefully considered. Investigators found that black older women scored lower than white older women after controlling for education and age (Wood et al. 2006), while others found that the MMSE may not be a valid predictor of functional cognitive ability in Black women (Leveille et al. 1998) suggesting that racial differences may have an effect on MMSE scores. Where a person lives seems to influence MMSE scores. After controlling for age, education, and race, investigators showed that the neighborhood where one lives influences MMSE scores (Espino et al. 2001). In sum, careful attention to the demographic selection of subjects in regard to the specific research question is an important consideration to the overall research design.

Examining report of the underlying assumptions of the MMSE

We conducted a review of published studies in the Cumulative Index of Nursing and Allied Health Literature (CINAHL), PubMed, Biomed Central, and PsycINFO to determine if inconsistencies existed in the literature. Specifically, we reviewed the methods sections of research studies using the MMSE on reporting education, sensory status, and fluency in language. The enormity of papers published that use the MMSE meant that studies were selected using the following search criteria. First, all databases were searched using the term “MMSE” selected in “title”. Second, the options to select “research” or “research and review” articles were chosen. Third, we selected the age ranges “60 and older”. Last, we selected only available full-text English articles for review. Recommendations for screening for the assumptions did not regularly appear until the early 1990s. We therefore limited our search to the years 1998–2010 and yielded a total of 121 hits. We focused on inclusion and exclusion criteria related to the administration of the MMSE. After removing useable citations that appeared in more than one database (n = 32), 39 articles were excluded: 12 were review articles, 13 were commentaries or letters to the editor, 10 were on other topics, and 4 were not geriatric focused (entire sample less than 60-years-old) leaving a final sample of 50 articles. Next, information was collected on each variable of interest: (1) education, (2) sensory status, and (3) language fluency. For the purposes of this review, we assumed that mention of “neurological” or “neuro” assessment was sufficient for indicating that an assessment of the subject’s sensory system occurred. Each article was reviewed for the presence of each variable (Table 2).

Table 2.

Review on reporting three assumptions for interpreting a valid MMSE exam score (1998–2010)

Study	Education screening	Sensory screening	Language screening	Other
Ryu and Kwon (2010)	Education and age matched	Neurologist evaluated all subjects	Administered Korean version K-MMSE	N = 234 people with Parkinson’s disease (age 71.1 ± 8.2) 166 women
Jervis et al. (2010)	Education (mean 10.5 years SD = 3.6)		Language proficiency assessed	N = 137 American Indian Elders Test administered by tribal member
Dong et al. (2010)	Education screening		Aphasia	N = 100 post stroke patients
Shigemori et al. (2010)		Neurological exam that included assessment of hearing and visual capacity		N = 30,895 consecutive outpatients with dementia in a Japanese dementia clinic
Morgado et al. (2010)	0–2 years, 3–6 years More than 6 years of education	Visual-constructive ability	Language fluency assessed	Sample 1N = 135 Sample 2N = 411 Compared 2 samples over time from same region
Wiig et al. (2010)		Neuropsychological screen		N = 60 (37 women), mean age 75.9, SD = 7.1
Wouters et al. (2010)				Used previously reported MMSE scores (secondary data), N = 1,566
Kovacevic et al. (2009)	At least 6 years of education	Visual and auditory systems assessed	Boston Naming Test used to assess language ability	Used ADNI data, N = 269 people with MCI
Guerrero-Berroa et al. (2009)	Assessed under 8th, 9th–12th or over 12 years			Prospective longitudinal study, N = 505 nursing home residents
Nazem et al. (2009)	Average education level 15.7 ± 3.6 years			Convenience sample N = 131. Trained RA’s administered the MMSE
Nelson et al. (2009)	Adjusted for total years of education			Used previously reported MMSE scores (secondary data), N = 5,813
de Jager et al. (2009)	Controlled for years education			Prospective cohort study N = 119
Brugnolo et al. (2009)	3 to 18 years included	“Other neuropsychological tests performed”		Mulit-center cross sectional study (N = 524)
Carcaillon et al. (2009)	Education level reviewed			Prospective population based three-city study (N = 1,516). MMSE administered by trained nurse or psychologist
Pang et al. (2009)	Educational level 8.4 years ± 2.1		Used professional Interpreters	Pilot study (N = 48)
Tiwari et al. (2009)	20–no education 20–5th grade or above	Senses all intact, speech, hearing, vision	Used translated version	N = 40 (20 illiterate and 20 literate)
Rajji et al. (2009)	Education level reported	Conducted “physical exams on all participants”		N = 447 depressed older adults recruited from 4 previous studies
Mamikonyan et al. (2009)	Age and education adjusted	Excluded those with color blindness	Excluded those unable to understand neuropsychological tests	N = 106 people with Parkinson’s disease
Han et al. (2008)	Educational level less than ≤6, 7–12, ≥13	Neuropsychological tests performed	Used translated version	Random sample (N = 997)
O’Bryant et al. (2008)	At least 16 years of education	Conducted “neuropsychological exams on all subjects”		N = 1141 consecutively enrolled subjects
Pezzotti et al. (2008)	Age and education adjusted for those who scored less then 24		Administered the Italian version of the MMSE	N = 59 general practitioners N = 317 subjects screened with the MMSE
Evans et al. (2008)	Education assessed		Language Assessed and test administered in Spanish or English	Pilot study N = 22 age (60–95) Mexican American nursing home residents
Yamashita et al. (2007)			Administered the Japanese version of MMSE	Quasi-experimental design, N = 14 (11 women), mean age 84.5
Mystakidou et al. (2007)	Completed primary school	Communicate effectively (hear and speak)	Administered the Greek version of the MMSE	Questionnaires baseline and 3 days later (N = 130)
Onishi et al. (2007)		Excluded those with hearing and writing difficulties		Between groups analysis (n = 792)—control, teaching hospital, long-term care, outpatient clinic, nursing homes.
Squitti et al. (2006)		Neurological testing completed		(N = 53), 28 AD patients matched to controls
Pachana et al. (2006)	Educational level assessed (59 % completed high school)	Hearing aid use		Pilot study (N = 46) of veterans in New Zealand
Chow et al. (2006)	Excluded if educational level not known	Excluded those with known visual impairment	Excluded individuals who did not speak English	Retrospective case–control (N = 89)
Noale et al. (2006)	Mental status total score (TS) were age and education adjusted	Vision (retinal photography)	Administered the Italian version of the MMSE	Used previously reported MMSE scores (secondary data) (N = 5,632)
Wood et al. (2006)	Age and education adjusted			Cross sectional N = 414 older Black and White women
Niwa et al. (2006)		Vision (retinopathy)		N = 92
Cullen et al. (2005)	Education adjusted			Cross sectional N = 1,115
Tombaugh (2005)	Average of 10.94 (SD = 3.90, range = 0–25) won 2010 years of education	Neuropsychological tests conducted	Administered in English	N = 756, 5-year follow up retest study
Huppert et al. (2005)	Age and education adjusted	Hearing and vision assessed	Language ability assessed	N = ~2,500 random sample
Koch et al. (2005)				N = 94 pairwise comparisons
Kliegel and Sliwinski (2004)	Education level assessed	Sensory system assessed		Study of 91 centurions
Salmon (2004)				Prospective longitudinal study, N = 165
Heinik et al. (2004)	Controlled for age and education	All subjects underwent “psychiatric and medical” testing		114 community dwelling elders examined by geriatric psychiatrists. Patient evaluation described elsewhere
Cazzaniga et al. (2003)	Age and education adjusted			N = 182 consecutive patients with probable AD. “MMSE administration methods cited in previous studies”
Schramm et al. (2002)		Clinical, neuropsychological, and apparative tests completed		N = 123 outpatient memory clinic. Age 44–90 (mean 69)
Nguyen et al. (2002)	Education level assessed	Hearing, vision assessed	Literacy and language skills assessed. MMSE administered in English or Spanish	Secondary data N = 1,759
Espino et al. (2001)	Education 0–8 years and >9	Sensory impairment visual and hearing	Exam language adjusted	827 community dwelling older adults—group analysis
Dufouil et al. (2000)	Educational level included in analysis	Included individuals with sensory deficit		Longitudinal study of cognitive decline
Maki et al. (2000)	Education level 7.3 (± 2.0)	Visual and auditory assessment completed	Verbal fluency assessed	N = 662 elderly subjects from the general population in Japan
Barbarotto et al. (2000)	Education 0–13 years (mean 5.5, SD = 3.6)			N = 27
Manos (1999)		Neuropsychological tests performed		N = 84 consecutive outpatient visits
Zlotogorski et al. (1999)	Mean education level 11.2 years			N = 31, ambulatory people with DAT mean age 70 (54–89)
Leveille et al. (1998)	Educational level assessed			N = 3,585, community dwelling women
Bauco et al. (1998)	Education level assessed			N = 101 centurions in Rome
The Medical Research Council Cognitive Function Ageing Society (1998)		Neurological tests completed		2 years of interviews on people who were 65 or older

Fourteen (28 %) of the 50 research articles reported all three assumptions: education, sensory assessment, and language (Table 2). Twelve (24 %) articles reported two of the three criteria; 21 (42 %) articles reported one of the criteria; and three (6 %) articles did not report any of these inclusion or exclusion criteria. Educational screening was the most commonly reported criterion 38 (76 %), followed by sensory status 29 (58 %), and language proficiency/screening 20 (40 %). The paucity of studies that clearly report meeting all three assumptions raises serious questions concerning the validity of research findings in which the underlying assumptions of cognitive testing (including the MMSE) have not been considered or reported.

Our review has some limitations. Studies included in the review may have screened for the assumptions but did not report them. A second limitation is that only studies with MMSE in the title were included in the review. For example, in the database “PubMed” there are 5,224 hits with the search term ‘MMSE’ and 581 hits with the search term ‘Mini Mental State Exam’. Thus, our search examining only studies with ‘MMSE’ in the title approximately represents from 2 to 20 % of all studies using the MMSE as a measurement instrument. Notwithstanding these limitations, we observed that there was consistent under reporting of the assumptions that underlie the MMSE.

Recommendations

The following recommendations are proposed for improving the validity of future research using the MMSE:

1. When using the MMSE, screen for and clearly report the sensory status (hearing & vision), educational level, and language fluency of study participants (Tombaugh and McIntyre 1992). Report that the assumptions were assessed with inclusion and exclusion criteria.

2. Carefully interpret MMSE scores in people over the age of 75, with less than an 8th grade education, or with suspected sensory loss. Consider using an age and education adjusted formula such as: MMSEadj = Raw MMSE − (0.471 × [education − 12]) + (0.131 × [age − 70]) (Mungas et al. 1996).

3. Consider including one or more of the severely cognitively impaired categories in research. More stratification (moderately severe, severe, or very severe) may improve the sensitivity and specificity of the MMSE in screening for dementia.

4. Consider using the MMSE in conjunction with other measures when discussing specific type or severity of dementia. Investigators should remain mindful that the MMSE was not intended to be a diagnostic tool for determining the cause of cognitive impairment.

5. Report that the assumptions were met in the original data set when using secondary data. Report this as a limitation if unable to determine if the assumptions were met.

6. Controlling for demographic covariates such as age, education, race, and neighborhood (or place of residence) will strengthen the study design.

Discussion

In summary, we propose recommendations for improving the validity of the implementation of the MMSE. These recommendations generally apply to many cognitive scales used in practice or research. The assumptions for improving validity may be inadvertently overlooked when reporting methods. The research literature could be missing important findings if tools are not being used correctly or the validity of research findings is open to question if tools are being used correctly and not reported as such. Subjects such as Mrs. Gonzales may be enrolled or excluded from studies resulting from inappropriate use of cognitive scales such as the MMSE. Future studies with expanded inclusion criteria are needed to assess the validity of the current findings.

Researchers and clinicians who use the MMSE (or any scale used in older adults) should be aware of limitations, assumptions, and the intended use of each. Clearly reporting that the assumptions for an exam score were met helps to assure that the findings are indeed valid.