Development of a Rapid Screening Instrument for Mild Cognitive Impairment and Undiagnosed Dementia

Abstract

Mild cognitive impairment (MCI) often presages development of Alzheimer’s disease (AD). We recently completed a cross-sectional study to test the hypothesis that a combination of a brief cognitive screening instrument (Mini-Cog) with a functional scale (Functional Activities Questionnaire; FAQ) would accurately identify individuals with MCI and undiagnosed dementia. The Mini-Cog consists of a clock drawing task and 3-item recall, and takes less than 5 minutes to administer. The FAQ is a 30-item questionnaire completed by an informant. In addition to the Mini-Cog and FAQ, a traditional cognitive test battery was administered, and two neurologists and a neuropsychologist determined a consensus diagnosis of Normal, MCI, or Dementia. A classification tree algorithm was used to pick optimal cutpoints, and, using these cutpoints, the combined Mini-Cog and FAQ (MC-FAQ) predicted the consensus diagnosis with an accuracy of 83% and a weighted kappa of 0.81. When the population was divided into Normal and Abnormal, the sensitivity, specificity and positive predictive value were 89%, 90%, and 95%, respectively. The MC-FAQ discriminates individuals with MCI from cognitively normal individuals and those with dementia, and its ease of administration makes it an attractive screening instrument to aid detection of cognitive impairment in the elderly.

INTRODUCTION

Alzheimer’s disease (AD) affects 10% of the population aged 65 and over [30]. Prior to the development of overt dementia, many individuals experience milder symptoms classified as mild cognitive impairment (MCI), which is often a prodromal stage of AD. Longitudinal studies have estimated that approximately 3%–19% of the elderly suffer from MCI, and that approximately 11%–33% convert from MCI to dementia over a 2-year period [15]. This large range of estimates stems partly from the fact that there is no universally accepted definition of MCI. However, MCI is generally understood to involve a cognitive complaint and evidence of cognitive decline, but without significant impairment of daily activities. In contrast, AD is characterized by substantial cognitive deficits in at least two domains (including memory) of sufficient severity to impair normal activities [26,28].

It is thought that there is widespread under-ascertainment of AD, with estimates that fewer than 50% of cases are diagnosed, and only approximately 25% are treated [30]. Under-ascertainment of MCI is presumably greater still, but accurate data are lacking. Although there is no current treatment to stop the progression of AD, drug treatments can improve dementia symptoms, and ideally, these treatments should be employed at the earliest stages of disease in order to sustain individuals at their highest level of functioning. While current drug treatments have not proven effective in preventing conversion of MCI to dementia, individuals with MCI benefit symptomatically from the use of cholinesterase inhibitors with some evidence for beneficial effects on disease progression [25]. Moreover, numerous investigational agents are under study as potential disease-modifying effects that may prevent or slow progression of AD. Given the likely under-ascertainment of MCI, and the requirement for early detection for application of new therapeutics, simple and effective tests are needed to screen for cognitive impairment among the elderly.

METHODS

Study population

We used two primary sources for the individuals included in this study. Patients without prior history of dementia were identified in the outpatient Geriatric Medicine Clinic at Wesley Woods Center of Emory University School of Medicine (n = 102). Geriatricians referred patients if they suspected possible cognitive deficits, or if patients or family expressed concern about cognitive decline. Additional participants were collected prospectively from individuals who were being enrolled as cognitively intact controls or possible MCI participants in a registry of research volunteers at the Emory Alzheimer’s Disease Research Center (n = 76). Participants were eligible for the study if they had a valid Mini-Cog score, a clinical diagnosis, and a Functional Activities Questionnaire (FAQ). Altogether, 204 participants had a Mini-Cog and a clinical diagnosis, but we only included the 178 eligible participants that also had the FAQ. An additional 26 subjects had the Lawton-Brody IADL [19] instead of the FAQ, and these individuals were excluded from the analyses. Six additional eligible individuals were excluded due to missing FAQ data because we were unable to corroborate functional information from a reliable informant. All study procedures were conducted with approval and in accordance with the ethical standards of Emory University’s Institutional Review Board.

Mini-Cog administration and scoring

There are a variety of methods of conducting the Mini-Cog test. In our case, the tester recited 3 words (apple, penny, table) and then had the patient repeat them verbally. The patient was instructed that he/she should try to remember the words because they would be asked to recall them later. Next, the patient was presented with a pre-drawn circle on a piece of paper and told, “I want you to imagine that this circle is the face of a clock. Place all the numbers on the clock and set the time to ten minutes after eleven.” After the clock was completed, the patient was asked to recall the 3 words they were given earlier. There was no specific time limit for the clock drawing test (CDT), but the task typically took less than 5 minutes. The three-word recall and CDT were each worth 3 points. Each correctly recalled word was given one point, and the CDT was scored as follows:

• 1 point: All numbers present with no omissions, duplications, or superfluous markings.

• 1 point: The numbers 12, 3, 6, 9 were on or adjacent to quadrant boundaries. All numbers were upright in the pre-drawn circle near the edge.

• 1 point: Hands indicated 11:10 by placement and proportion. Scorer was able to tell the intended time without knowing what time the participant was told to draw.

Functional activities questionnaire (FAQ)

The FAQ [27] was completed by a reliable informant, generally a spouse, other family member, or close friend. The questionnaire is self-explanatory, and informants were asked to complete the form while the research participant was performing other tasks. Time required to complete the FAQ was not collected as this was completed without the involvement of staff. The completed form was collected, scored, and recorded. Performance on each of ten items reflecting instrumental activities of daily living (IADL) was reported as ‘normal’, ‘has difficulty but does by self’, ‘requires assistance’, or ‘dependent’ (scored 0 to 3, respectively). A score of 30 represents maximal dependence, and a score of 0 represents complete independence. Pfeffer and colleagues originally suggested the following cutoffs: normal functioning ~3, questionably affected ~12, mildly affected ~15, moderately affected ~23, and severely affected ~29 [27].

Cognitive testing battery

All participants were tested with a common battery of neuropsychological tests, including the Mini-Mental State Examination (MMSE) for global functioning and two tasks in each of five cognitive domains.

Memory: CERAD Word List [23] and Brief Visuospatial Memory Test-Revised [3].

Language: Boston Naming Test [18] and Controlled Oral Word Association [4]

Attention: Digit Span Forward [35] and Trail-Making, part A [1]

Executive Function: Trail-Making, part B [1] and Clock Drawing Test [14]

Visuospatial ability: Digit-Symbol [34] and Judgment of Line Orientation [5]

Results of the cognitive tests were converted to z-scores using standard age-specific norms. For the CER-AD word list test, norms were also adjusted for education. The CDT portion of the cognitive battery was scored differently than the Mini-Cog by using an elaborated 13-point scale with published norms [14]. The neuropsychologist was not blinded to Mini-Cog as 3 item recall (in MMSE) and CDT were included in the testing, but the FAQ was not included in the neuropsychological assessment.

Consensus diagnosis

Two experienced behavioral neurologists (AIL, JJL) individually reviewed clinical history of all participants. The neurologists did not have access to the Mini-Cog or FAQ results, but they did review the cognitive testing results and the neuropsychologist’s (FCG) impression. All participants were diagnosed as Normal, MCI, or Dementia. The classification of MCI was not limited to those with memory impairment (MCI-amnestic), and includes individuals who demonstrated impairments in other single domains or multiple domains. For those demonstrating difficulties in multiple cognitive domains, designation as either MCI or Dementia was generally driven by review of records to obtain a sense of the individual’s functional independence. The neurologists agreed on initial diagnosis for 90% of cases (184/204), and the weighted kappa was 0.90. In case of differing diagnoses, a consensus diagnosis was reached through discussion among all clinicians.

Determination of optimal cutpoints

We used a classification tree algorithm (CART, or Classification and Regression Trees [9]) implemented in R [33] to determine the best cutpoints for the Mini-Cog and FAQ to predict Normal, MCI, and Dementia diagnoses. The CART algorithm selects optimal cutpoints by recursively partitioning the observations to achieve the maximum reduction in the Gini index, a measure of disparity between the predicted and observed values. In our study we penalized misclassification across two ordered categories (e.g., Dementia classified as Normal) twice as much as misclassification across adjacent categories (e.g., Dementia classified as MCI). We also considered possible modifying effects of age, sex, education, and race to determine whether optimal cutpoints might differ by these variables.

Statistical analyses

Three-by-three tables, comparing gold standard diagnosis (consensus diagnosis) vs. prediction based on cutpoints for Mini-Cog and FAQ scores as determined by the tree algorithm, were evaluated via calculation of accuracy and weighted kappa statistics. In addition, we used a polytomous logistic model, with no proportional odds assumption, in which the outcomes were the true diagnostic categories of Normal, MCI, and Dementia. We used this model to: 1) compare other cognitive tests with the Mini-Cog and FAQ using model fit to evaluate which tests best predicted the consensus diagnosis; and 2) to evaluate which demographic variables were significant predictors of outcome when cognitive tests were already in the model. Finally, analyses were conducted using only two diagnostic categories, Normal vs. Abnormal (MCI and Dementia combined). Cut-points from the original analyses were used to separate Normal from Abnormal. Restriction of the data to two diagnostic categories made possible calculation of standard statistics such as sensitivity, specificity, and positive predictive value.

RESULTS

Descriptive characteristics

Table 1 presents descriptive statistics for the 178 participants. The overall average age of subjects was 77. The subjects were 22% African-American, 72% female, and 37% had a high school education or less. The proportions diagnosed as Normal, MCI, and Dementia were 38%, 30%, and 32%, respectively. Groups differed significantly by age, education, and race. A higher percentage of females were present in the Dementia group than Normal, but the MCI group did not differ from the Normal group for sex. As expected, Dementia, MCI, and Normal groups differed significantly on Mini-Cog, MMSE, and FAQ scores after adjusting for demographic covariates.

Table 1.

Demographics and test results for three groups of participants

	Normal, n = 68	MCI, n = 53	Dementia n = 57
Age	73	77*	81*
Years education	16.0	14.7*	12.5*†
Percent female	63%	73%	81%*
Percent non-white	9%	23%*	37%*
Mini-Cog (0–6)	5.1	3.8*	1.9*†
MMSE (0–30)	29.0	26.9*	19.1*†
FAQ (0–30)	0.5	5.1*	18.4*†

^*Indicates that the either demented or the MCI were significantly (P < 0.05) different from normal group, as evaluated via linear regression with categorical variables for MCI and demented groups, each compared separately to the normal group. Regressions for Mini-cog, MMSE, and FAQ included variables for age, gender, education, and race in addition to the categorical variables for MCI and demented.

^†Indicates that the demented group was significantly (P < 0.05) different from MCI group.

Determination of Mini-Cog and FAQ cutpoints and comparison of true diagnosis versus predicted

Table 2 shows the cutpoints used for the Mini-Cog and FAQ to classify participants into diagnostic categories. These cutpoints were chosen via CART to optimize overall agreement between test results and the final diagnosis. Additional analysis of the relationship between separate components of the Mini-Cog and FAQ with age and education were examined by univariate Spearman correlations. Correlations between education and 1) the clock score and 2) the 3 word recall were 0.44 and 0.32, respectively (both p < 0.0001). Univariate Spearman correlations between age and 1) the clock score and 2) the 3 word recall were −0.42 and −0.29, respectively. The univariate Spearman correlation between the FAQ and 1) age and 2) Mini-Cog score were 0.37 and −0.36 respectively. In the CART algorithm, age was the only demographic covariate that provided any increase in accuracy after Mini-Cog and FAQ scores were used, but the increase was small (85% vs. 83%) and it complicated the classification criteria, increasing the number of discrete categories from seven to eleven. For increased simplicity, we did not include age with the Mini-Cog and FAQ scores for determining optimal cutpoints. Among those with significant functional limitations (FAQ > 12), only those who scored a perfect 6 points on the Mini-Cog were classified as MCI. All others in this group were classified as Dementia. Classification of subjects based on functional (FAQ) and cognitive (Mini-Cog) axes are shown in Fig. 1.

Table 2.

Table 2a. Cutpoints for Mini-Cog* and FAQ* to classify participants
Normal	MCI	Dementia
FAQ 0–2; MC 4–6	FAQ 0–2; MC 0–3
	FAQ 3–9; MC 0–6
	FAQ 10–12; MC 3–6	FAQ 10–12; MC 0–2
	FAQ ⩾13; MC=6	FAQ⩾13; MC 0–5

Table 2b. Cutpoints for MMSE* and FAQ* to classify participants
Normal	MCI	Dementia
FAQ 0–2; MMSE 27–30	FAQ 0; MMSE 0–26	FAQ 1–30; MMSE 0–25
	FAQ 1–2; MMSE 26
	FAQ 3–30; MMSE 26–30

Mini-Cog scored 0–6 points with 6 the best score; FAQ scored 0–30 points with 0 the best score.

MMSE scored 0–30 points with 30 the best score; FAQ scored 0–30 points with 0 the best score.

Fig. 1.

A. Distribution of cases along functional (FAQ) and cognitive (Mini-Cog) axes. Classification based on the MC-FAQ screener can be read off of a simple grid. Cases falling within white cells are Normal, those in gray cells are MCI, and those in black cells are Dementia. B. Distribution of cases along functional (FAQ) and cognitive (MMSE) axes. Cases falling within white cells are Normal, those in gray ells are MCI, and those in black cells are Dementia.

Table 3a shows the agreement between classification based on the Mini-Cog and FAQ (MC-FAQ) and the clinical diagnosis. The accuracy was 83%, and the weighted kappa was 0.81. For comparison, Table 3b shows the agreement between classification based on the MMSE and the FAQ (MMSE-FAQ). The accuracy was 85% and the weighted kappa was 0.83. Importantly, among true MCI, 74% (39/53) were correctly classified as MCI by the MC-FAQ screener compared to 68% (36/53) by the MMSE-FAQ screener.

Table 3.

Table 3a. Agreement* of screening test (Mini-Cog + FAQ) and consensus diagnosis
Consensus diagnosis	Diagnosis based on screening test (Mini-Cog + FAQ)
	Dementia	MCI	Normal
Dementia	48	9	0
MCI	2	39	12
Normal	0	7	61

Table 3b. Agreement* of screening test (Mini-Cog + FAQ) and consensus diagnosis
Consensus diagnosis	Diagnosis based on screening test (MMSE + FAQ)
	Dementia	MCI	Normal
Dementia	52	5	0
MCI	6	36	11
Normal	0	5	63

Cutpoints for Mini-Cog and FAQ based on tree algorithm; accuracy 83%, weighted kappa 0.81.

Cutpoints for MMSE and FAQ based on tree algorithm; accuracy 85%, weighted kappa 0.83.

Polytomous logistic regression analyses

The polytomous logistic regression model using diagnosis as the outcome and Mini-Cog, FAQ, age, race, gender, and education as predictors indicated that only Mini-Cog, FAQ, and age were significant predictors of clinical diagnosis. This was consistent with the classification tree, and suggests that once these predictor variables were used, no additional predictive information was gained by considering race, gender, or education.

Table 4 shows results of polytomous logistic regression analyses for a variety of models using different cognitive tests as predictors. Higher values indicate a better fitting model. In the cognitive battery, executive and memory domain tests were better predictors of diagnosis than visuospatial, language, and attention domain tests. They were also superior to the Mini-Cog when it was considered individually. In this model, both the Mini-Mental State Exam (MMSE) and the FAQ alone performed better than the Mini-Cog alone. The best tests were the MMSE-FAQ and the MC-FAQ, with the MMSE-FAQ yielding a better fitting model than MC-FAQ. The superior performance of the MMSE may reflect the greater range of continuous variables allowed by the MMSE compared to Mini-Cog for the polytomous logistic regression analysis (0–30 vs. 0–6, respectively). However, after deriving optimum cut-points via the classification tree algorithm, both pairs of tests gave similar results for accuracy and weighted kappas (0.83 and 0.81 for MC-FAQ, and 0.85 and 0.83 for MMSE-FAQ; Tables 3a and 3b). These findings indicate that the MMSE may offer a small advantage over the Mini-Cog as a cognitive screen when each is paired with the FAQ. However, this came at the expense of time, as the MMSE took about 10–15 minutes to administer, while the Mini-Cog required 3–5 minutes.

Table 4.

Ability to predict normal, MCI, and dementia based on a polytomous logistic model

Predictive test(s) in model	Logistic model likelihood*
Attention	127
Visuospatial	133
Mini-Cog	142
Language	154
Executive	173
Memory	181
FAQ	203
MMSE	204
Mini-Cog + FAQ	232
MMSE + FAQ	257

The model likelihood is a chi square statistic with 5 degrees of freedom (6 for the last two models with two tests). The model includes 4 demographic covariates (age, race, sex, education). A higher model likelihood means a better fitting model. An improvement in model likelihood of 3.8 or greater (6.0 or greater for models with two predictive tests) is a statistically significant improvement at the P = 0.05 level. All cognitive tests were highly significant predictors. Model likelihood with demographic covariates alone was 73.

Based on average of z-scores for two tests in each domain.

Sensitivity, specificity, and positive predictive value

We next examined the ability of the MC-FAQ and MMSE-FAQ to distinguish between ‘normal’ and ‘not normal’. This perspective is appropriate when considering a general elderly population, in which the true diagnosis is not known but we wish to detect possible cases of either MCI or undiagnosed Dementia for further work-up. In this case, true Dementia cases and true MCI cases were collapsed into a single ‘Abnormal’ category, and the same was done with the test results. Using the cutpoints shown in Table 3 for Normal vs. Abnormal, the sensitivity (proportion of true positives who test positive) and specificity (proportion of true negatives who test negative) of the MC-FAQ screener were 89% (98/110) and 90% (61/68), respectively, while the positive predictive value (proportion of test positives who are truly positive) was 95% (98/103). For comparison, a similar analysis was performed for the MMSE-FAQ, using optimal cutpoints determined by the classification tree algorithm. The sensitivity, specificity, and positive predictive value of the MMSE-FAQ were 90% (99/110), 93% (63/68), and 96% (99/103), respectively. These results suggest that a combination of either MMSE or Mini-Cog with the FAQ can effectively identify individuals with both Dementia and MCI, with the MMSE-FAQ performing slightly better than the MC-FAQ overall. However, administration of the MMSE required 2–5 times longer than the Mini-Cog, and the MC-FAQ more accurately classified MCI cases (74%) than the MMSE-FAQ (68%).

DISCUSSION

The primary purpose of our study was to determine whether a combination of a brief cognitive screening test (Mini-Cog) along with a measure of functional ability (FAQ) would effectively discriminate between cognitively normal elderly, those with MCI, and those with dementia. Since both cognitive performance and functional abilities are implicit in the concepts of MCI and dementia, we reasoned that adding the FAQ to a cognitive screener would improve the ability to detect these conditions. By using simple cutpoint rules for the MC-FAQ, we were able to correctly classify 83% of our subjects as Normal, MCI, or Dementia. Furthermore, the MC-FAQ had a positive predictive value of 95% for discriminating between cognitively normal elderly individuals from those with MCI or dementia, and it was able to correctly classify 74% of individuals with MCI. The combination of the MMSE and FAQ had a slightly better ability to predict the correct diagnosis overall (85% vs. 83%), but at the cost of double or triple the time necessary to administer the test than the Mini-Cog. In addition, the MMSE-FAQ was slightly less effective than the MC-FAQ in classifying individuals with MCI (68% vs. 74%). These results suggest that the administration of the MC-FAQ may be an effective and efficient approach to widespread screening for cognitive impairment.

Rapid advances in basic and clinical research on AD are expected to yield meaningful improvements in the treatment of patients with this devastating illness. However, effective methods must be developed to identify those individuals in the earliest stages of disease to allow maximal preservation of cognitive abilities. Development of advanced imaging techniques and effective blood or cerebrospinal fluid biomarkers hold promise for aiding early detection, but these approaches have major limitations. For example, imaging methods using new positron emission tomography (PET) tracers to detect development of neuropathological changes [22, 29] are expensive and inaccessible to large portions of the population. Use of biomarkers often require invasive testing as well as having the limitations of costs and time required to return results. Moreover, considerable research will be required to establish the utility of any radiological and biochemical biomarker in tracking true disease progression.

Currently available screening instruments are effective at detecting individuals with overt dementia, but we are not aware of any established clinical screening instrument for identifying individuals with MCI. A number of diagnostic approaches have been evaluated, and these have generally focused on identifying the best cognitive test or battery of tests for detecting MCI [12, 20,28,37]. While some of these methods have shown encouraging results with sensitivities and specificities ranging from 80% to over 95%, most involved rather complex neuropsychological tests that are not feasible to deploy in the primary care setting or community-based screening efforts. Another key limitation in all of these reported methods is that the cognitive testing is expected to require 15 to 40 minutes to administer. Practical limitations in current delivery of ambulatory medical care offer compelling motivations for the development of a much shorter screening instrument to detect cognitive impairment. The 2004 National Ambulatory Medical Care Survey reported that the average duration of an office visit with a primary care physician was 17.9 minutes [17]. Other estimates suggest that the duration of office visits may be only 13.3 minutes [16]. The most widely accepted cognitive screening test, the MMSE [13], typically takes about 10 minutes to administer, and time has been reported by physicians to be the most common deterrent to performing the MMSE [32]. The MMSE, used alone, has also previously shown to be neither sensitive nor specific in screening for MCI [28]. Large scale screening for MCI and mild dementia will clearly require the development of a simple, effective, and very brief instrument.

The Mini-Cog has been evaluated previously and shown to have good sensitivity and specificity for dementia [7,21]. It has often been identified as a good option because it is comparable to the MMSE in its ability to predict dementia, and it can be administered in an average of 3.2 minutes [6,10,36]. We did not specifically record the time required to administer the Mini-Cog, but our experience parallels the original report of its brevity [6]. However, the Mini-Cog’s sensitivity for detecting MCI is only 55% [8]. In addition to cognitive screening tests, a number of informant-based functional questionnaires have been evaluated as a means of detecting cognitive decline. The FAQ, which we used in this study, was previously identified by the Agency for Health Care Policy and Research as an effective means of identifying demented individuals with sensitivities and specificities in the 85–90% range [11]. Although the FAQ has not been closely examined as a screener for MCI, greater informant-reported functional impairment was associated with a higher risk of conversion from MCI to AD during a two-year follow-up period [31]. A similar increased risk of conversion from MCI to AD was also recently reported using the Lawton and Brody IADL scale [19,24]. These findings confirm previous reports that have suggested that cognitively impaired individuals tend to overestimate their functional abilities [2], and suggest that a functional assessment will enhance the identification of individuals with very mild cognitive deficits.

Inclusion of a functional assessment questionnaire, such as the FAQ, in screening for MCI requires that a reliable informant be available and willing to complete the questionnaire. Our experience in this study suggests that this requirement will not pose a significant impediment to using the MC-FAQ as a screening instrument. For 97% of subjects, a spouse or other family member was readily available to provide a functional assessment (either the FAQ or the LB- IADL), and in most instances, they routinely accompanied the elderly relative to doctor’s appointments. In cases of elders who live alone and visit clinicians unaccompanied, the need for an informant to complete the FAQ might pose difficulties, but alternatives, such as telephone interviews or on-line completion of the questionnaire, may be feasible. Nevertheless, in most cases, an important practical consideration for the deployment of the MC-FAQ into primary care settings is that the FAQ can be collected without infringing on the time available for interaction between patient and clinician. Finally, despite the addition of the FAQ “functional axis” to the Mini-Cog “cognitive axis”, scoring of the combined MC-FAQ remains extremely straightforward (see Figure 1), and we anticipate that formal measures will demonstrate very high inter-rater reliability.

As in most evaluations of screening instruments, the performance of the MC-FAQ in the current study is likely to have benefited from the high rate of affected individuals in the test population, and our results will need to be confirmed in the general population. Nevertheless, the current findings, along with key practical considerations discussed above, suggest that the MC-FAQ may provide an important and effective screening instrument to improve detection of individuals with MCI as well as undiagnosed mild dementia.