Which part of the Quick mild cognitive impairment screen (Qmci) discriminates between normal cognition, mild cognitive impairment and dementia?

Abstract

Introduction: the Qmci is a sensitive and specific test to differentiate between normal cognition (NC), mild cognitive impairment (MCI) and dementia. We compared the sensitivity and specificity of the subtests of the Qmci to determine which best discriminated NC, MCI and dementia.

Objective: the objective was to determine the contribution each subtest of the Qmci makes, to its sensitivity and specificity in differentiating MCI from NC and dementia, to refine and shorten the instrument.

Methods: existing data from our previous study of 965 subjects, testing the Qmci, was analysed to compare the sensitivity and specificity of the Qmci subtests.

Results: all the subtests of the Qmci differentiated MCI from NC. Logical memory (LM) performed the best (area under the receiver operating curve of 0.80), registration the worst, (0.56). LM and verbal fluency had the largest median differences (expressed as percentage of total score) between MCI and NC, 20 and 25%, respectively. Other subtests did not have clinically useful differences. LM was best at differentiating MCI from NC, irrespective of age or educational status.

Conclusion: the Qmci incorporates several important cognitive domains making it useful across the spectrum of cognitive impairment. LM is the best performing subtest for differentiating MCI from NC.

Introduction

As time is limited in clinical practice, short cognitive screens help to improve diagnostic efficiency and are useful in detecting and quantifying cognitive impairment. One of the major challenges in cognitive testing has been the development of rapid screening tests to differentiate mild cognitive impairment (MCI) from normal cognition (NC). Tools, such as the Folstein MMSE [1] and standardised Mini-Mental State Examination (SMMSE) [2, 3], are useful in distinguishing NC and MCI from dementia, but take time to complete, and are less able to distinguish MCI from NC [4, 5]. Identifying MCI is important as it can be a prodrome to dementia [6] and allows earlier recognition of individuals at risk [7]. Although treatment options are limited, a diagnosis of MCI should prompt the search for reversible causes of cognitive impairment. The International Working Group on MCI suggested that population screening cannot be recommended at present, as there is insufficient evidence for sensitive and specific tools, including cognitive tests [7]. Few tools used for detecting MCI are specific for the condition, because they were developed as dementia screening tests [8]. Some, such as the Montreal Cognitive Assessment (MoCA) [9], the Alzheimer's Disease Assessment Scale-Cognitive section (ADAS-cog) [10] and the AB Cognitive Screen 135 (ABCS 135) [5], have shown improved sensitivity for detecting MCI when compared with the SMMSE.

The MoCA is widely used and valid in different clinical settings including Parkinson's disease, cerebrovascular disease [11] and Huntington's disease [12], but takes at least 10 min to perform. The ADAS-cog [10] also screens for MCI [13], but takes up to 45 min, requires trainings [14] and has ceiling effects, possibly limiting its usefulness [15, 16]. The addition of executive function and functional ability subtests has recently improved its sensitivity [16]. The ABCS 135 is more sensitive and shorter than the SMMSE at differentiating MCI from NC and dementia [5]. It is composed of five subtests, orientation, registration, clock drawing, delayed recall for words (DR) and verbal fluency (VF) for animals. The Qmci, the Quick mild cognitive impairment screening test, was developed to improve upon the ABCS 135 and is more sensitive and specific in differentiating MCI from NC [4].

Development of the Qmci

The Qmci was created from the ABCS 135, by reweighting the original subtests and adding a logical memory (LM) section. Previous analysis of the ABCS 135 subtests found that DR and VF were more sensitive than orientation, registration and clock drawing, in distinguishing MCI from NC [17]. LM, a verbal memory test, using immediate recall, was added because it is highly sensitive and specific in differentiating MCI from NC [18]. The original subtests had their absolute scores reduced, to allow for the introduction of LM, with the weightings of DR and VF increasing relative to the others. Orientation scores 10 points, registration 5, clock drawing 15, DR and VF 20 each and LM 30. The Qmci total score of 100 points is easier to use than a total score of 135, in the original ABCS 135.

The Qmci, therefore, has six subtests covering the following cognitive domains: orientation, working memory (registration), visuospatial/executive function (clock drawing), semantic memory (VF) and two episodic memory domains (DR and LM). The Qmci can be completed in 3–5 min, median time 4.24, and is more sensitive than the SMMSE and ABCS 135 in discriminating MCI from NC and dementia [4]. It is more clinically useful than these other tests, as it has greater median percentage differences between subjects with MCI and NC [4].

The primary objective of this study was to compare the sensitivity and specificity of different subtests of the Qmci in differentiating between NC, MCI and dementia. The secondary objective was to assess the effect of age and educational attainment on the sensitivity and specificity of individual Qmci subtests.

Methods

Subjects

Subjects were recruited between 2004 and 2010 from patients attending four memory clinics in Ontario, Canada. A total of 1,006 subjects were assessed, 53 were excluded, 965 individuals were included; 16% (n = 154) had MCI, 19% (n = 181) dementia and 65% (n = 630) NC. Normal controls, selected by convenience sampling, comprised exclusively of caregivers attending with the subjects, without symptoms of memory loss. Subjects were excluded if they were unable to communicate verbally in English (n = 12), if they had depression (n = 33), including 21 controls with subjective memory loss but NC, or if a reliable collateral history was unavailable (n = 8). Parkinson's disease and Lewy body dementia cases were excluded as they typically present with marked functional impairment and a different MCI syndrome [19].

Dementia was diagnosed based on NINCDS [20] and DSM-IV criteria [21] and was correlated with the Reisberg FAST scale [22]. The majority (78%) of dementia cases were mild (n = 141). Removing moderate and severe cases did not affect sensitivity [4]. As no consensus on diagnostic criteria exists [23], MCI was diagnosed clinically, by a consultant geriatrician following a comprehensive assessment of patients with recent, subjective and or corroborated memory loss without obvious loss of function. Assessment included comprehensive history, physical examination, laboratory screening, functional assessment, behavioural scores and depression screening (Geriatric Depression Scale, GDS, greater than seven [24]). No objective cognitive test was used in the classification of MCI. This in keeping with criteria previously proposed by the MCI Working Group of the European Consortium on Alzheimer's Disease (EADC) [25], but differs from others such as the International Working group on MCI [7] which suggests the use of objective cognitive testing. Cognitive tests were performed in random order, by trained raters, blind to the diagnosis and prior to the assessment.

The functional level was measured using the Quick Activity of Daily Living (Qadl) score, unpublished work, measuring basic and instrumental ADLs. Unless there was co-existing physical disability, all subjects with MCI had normal Qadl scores. Subjects with dementia varied, depending upon stage and physical disability. Behaviours were recorded using the Quick Behaviour score, unpublished work, which condenses 12 items from the Dysfunctional Behaviour Rating Instrument [26]. The most frequent behaviour reported was repetition, 81.9% for dementia, 82.1% for MCI, P = 0.36. Statistically significant differences were seen for social withdrawal, 47.5% for dementia versus 22.4% for MCI, P = 0.003, sleep disturbance (61.4 versus 46%, P = 0.05) and aggression (10.5 versus 1.5%, P = 0.038). Ethical approval was obtained and subjects provided consent.

Data analysis

Data were analysed using SPSS 16.0. Subgroup analysis was performed for age (greater and less than 75 years, to provide balance in sample size between groups) and for years of formal education (greater or less than 12 years, based upon UNESCO data [27]). Normality was tested using the Shapiro–Wilk test. The majority of the data were not normally distributed and were analysed using a Mann–Whitney U test. Normally distributed data were analysed using Student's t-tests. Pearson Chi-squared tests were used to establish the difference between the distributions when it was not possible to analyse differences in medians. Receiver operating characteristics (ROC) curves were constructed based upon the sensitivity and specificity of the Qmci subtests. Area under the curve (AUC) was calculated for each subtest and analysed for age and years of education. Nine subjects, without complete data, were excluded from this analysis. Test–retest reliability was demonstrated by measuring the Qmci on two separate occasions, 1 week apart, for a small sample of subjects, chosen by simple randomisation, n = 20. Pearson's correlation coefficient showed good test–retest correlation, 0.86.

Results

Figure 1 shows box plot distributions for each Qmci subtest, with median and inter-quartile range (IQR) scores for subjects with dementia, MCI and NC. The VF and LM subtests of the Qmci clearly distinguish between dementia, MCI and NC. Orientation and registration did not show a median difference between MCI and NC. All individual subtests had statistically significant differences, P < 0.001, in distributions between NC, MCI and dementia.

Figure 1.

Box plots distributions for each subtest of the Qmci showing the median and inter-quartile range scores for dementia (D), mild cognitive impairment (MCI) and normal cognition (NC).

Table 1 shows the median scores and IQR's for the Qmci subtests along with the P-value of the median difference between the scores of subjects with either MCI and NC or MCI and dementia. The overall median scores and differences are also shown for the Qmci as a whole. Although there were statistically significant differences between Qmci subtests scores, they were not all able to differentiate MCI and NC in a clinically useful way. The median difference in scores between MCI and NC was clinically useful for DR (four point difference), VF (four points) and LM (7.5 points). These differences, expressed as a percentage of the total score for each subtest, are 20% (four point difference out of a total score of 20), for DR, 20% for VF and 25% for LM. There was a median one point difference for clock drawing (6.66%), and no difference (0%), for orientation and registration, between MCI and NCs, suggesting that these three subtests are clinically less useful.

Table 1.

Qmci subtests: median scores and IQR (Q1 = 1st Quartile, Q3 = 3rd Quartile) by diagnosis, and P-value of the median difference between MCI and NC, dementia and MCI, along with AUC scores for SMMSE and the best performing Qmci subtest, LM, by age and education, for differentiating NC from MCI

Item	NC median (Q3-Q1 = IQR) (n = 630)	MCI median (Q3-Q1 = IQR) (n = 154)	Dementia median (Q3-Q1 = IQR) (n = 181)	P-value of the median diff between MCI-NC	P-value of the median diff between MCI-Dementia
Qmci total	76 (83–69 = 14)	62 (68–53 = 15)	36 (45–23 = 22)	P < 0.001	P < 0.001
Qmci subtests (score out of)
Orientation (10)	10 (10–10 = 0)	10 (10–9 = 1)	7 (9–5 = 4)	P < 0.001	P < 0.001
Registration (5)	5 (5–5 = 0)	5 (5–4 = 1)	5 (5–3 = 2)	P < 0.001	P = 0.002
Clock drawing (15)	15 (15–15 = 0)	14 (15–13 = 2)	11 (14–2 = 12)	P < 0.001	P < 0.001
Delayed recall (20)	16 (20–12 = 8)	12 (16–8 = 8)	0 (8–0 = 8)	P < 0.001	P < 0.001
Verbal fluency (20)	11 (13–9 = 4)	7 (9–6 = 3)	4 (6–2 = 4)	P < 0.001	P < 0.001
Logical memory (30)	20 (24–16 = 8)	12.5 (16–10 = 6)	8 (10–2 = 8)	P < 0.001	P < 0.001
Group (MCI and NC, n = X)	Test variables	Area under curve (95% CI)	Median diff MCI and NC (P-value)
Age ≤75 with education <12 years n = 127	SMMSE	0.65 (0.54–0.76)	29 (P = 0.011)
	LM	0.72 (0.62–0.82)	15.49a (mean) (P < 0.001)
Age ≤75 with education ≥12 years n = 449	SMMSE	0.66 (0.57–0.75)	29 (P < 0.001)
	LM	0.79 (0.73–0.86)	20 (P < 0.001)
Age >75 with education <12 years n = 71	SMMSE	0.64 (0.51–0.77)	28 (P = 0.034)
	LM	0.74 (0.62–0.85)	14.35a (mean) (P < 0.001)
Age >75 with education ≥12 years n = 127	SMMSE	0.55 (0.44–0.66)	29 (P = 0.350)
	LM	0.79 (0.71–0.88)	16.91a (mean) (P < 0.001)
Overall	SMMSE	0.67 (0.62–0.72)	29 (P < 0.001)
	LM	0.80 (0.76–0.84)	18 (P < 0.001)

^aNormally distributed data.

ROC curves in Figure 2a illustrate the sensitivity and specificity of the Qmci in differentiating NC from MCI, compared with the ABCS 135 and the SMMSE. The Qmci performs better in distinguishing NC from MCI with an AUC of 0.86, compared with the ABCS 135 (0.82), and the SMMSE (0.67). Taken in isolation, the LM component of the Qmci, scored higher than the ABCS 135 and SMMSE (AUC of 0.80 and 0.67, respectively). We also examined the individual subtests of the Qmci to assess their accuracy. The ROC curves in Figure 2b compare the ability of the subtests to discriminate between MCI and NC. The most accurate subtest is LM (AUC of 0.80), followed by VF (0.77), and DR (0.73). Registration (0.56), orientation (0.57) and clock drawing (0.66) were the least accurate subtests. The best performing SMMSE subtest was short-term memory (0.66), the worst registration (0.51).

Figure 2.

ROC curves illustrating the sensitivities and specificities of (a) Qmci, logical memory component (LM) of the Qmci, ABCS 135 and SMMSE in differentiating MCI from NC (b) Qmci subtests in differentiating MCI from NC and (c) Qmci subtests, in differentiating dementia from MCI. *Area under the curve (AUC).

The Qmci (total), ABCS 135, SMMSE and LM subtest had similar performance in differentiating MCI from dementia. LM, alone, performed particularly well with an AUC of 0.82. The AUC for the Qmci total score was 0.92 (95% CI: 0.89–95), suggesting it has similar accuracy, in differentiating MCI from dementia, to the ABCS 135 (AUC 0.91; 95% CI: 0.88–0.94) and the SMMSE (0.91; 95% CI: 0.86–0.94). Each of the subtests of the Qmci accurately distinguished MCI from dementia, (see Figure 2c). The best tests were orientation (AUC of 0.88) and DR (0.84). Registration was again the worst performing test (AUC of 0.64). The best performing SMMSE subtest was orientation (0.82) and the worst was registration (0.54).

Table 1 also shows AUC values for the SMMSE compared with the best performing subtest, LM. The AUC for the LM was superior to the SMMSE, in differentiating MCI from NC, irrespective of the educational level, or age (over or under 75 years) of subjects. The improved performance of LM over the SMMSE was more evident for the older age group (age over 75) who had over 12 years of formal education, AUC of 0.79 (95% CI: 0.71–0.88) versus 0.55 (95% CI: 0.44–0.66). There was a significant difference between the median scores for LM, for the MCI and NC groups, irrespective of age or educational status. This difference was not significant for the SMMSE for older people, >75 years with >12 years in education (P = 0.350).

Conclusion

The importance of MCI is only matched by difficulties in its diagnosis, particularly in its differentiation from NC. The Qmci can differentiate MCI from NC and is more sensitive and specific than the SMMSE and ABCS 135 in distinguishing MCI from NC and dementia [4]. The Qmci includes a battery of subtests, but not all differentiate MCI from NC in a clinically useful way. This study found that subtests with the greatest median differences between MCI from NC, expressed as a percentage of their total scores, were DR, VF and LM. LM, added to the original ABCS 135, improved the sensitivity of the test in differentiating MCI from NC and is the most useful subtest of the Qmci. Orientation, registration and clock drawing, as individual subtests, do not enhance the discriminating power of the tool to the same extent. These subtests have lower ceilings and are insensitive to early cognitive changes [17]. When age and education were taken into account, the best performing subtest, LM, was more accurate than the SMMSE in differentiating MCI from NC, suggesting that alone, it may be better at distinguishing MCI in the oldest and most educated subjects. All subtests could differentiate dementia from MCI and NC. None of the SMMSE subtests performed better than the complete SMMSE or LM.

From the results, we conclude that tests targeting episodic memory (DR and LM) best discriminate MCI from NC, whereas orientation is best for assessing dementia, allowing the Qmci accurately monitor disease progression. The remaining subtests, further enhance sensitivity, structure the test and may enhance its ability to identify MCI syndromes that convert to different dementia subtypes.

The strength of this study is that it included large numbers of patients with MCI and dementia, and that the tool was validated in a clinical sample in a busy memory clinic, increasing the generalisability of these results. A weakness is that it compares the Qmci to the SMMSE and ABCS 135, which are not gold standards for diagnosing MCI or dementia. No objective cognitive testing was used in the diagnosis of MCI which may also have led to bias although the diagnosis and criteria remain ill-defined [23]. The GDS, used to support a diagnosis of depression, is limited in advanced dementia [28], although the majority of subjects in this study were at an early stage. Subjects were only classified with MCI if there was no evidence of functional impairment. This may have created bias given that evidence suggests that up to 30% of subjects with MCI may have subtle impairment in instrumental ADLs [29]. Another limitation is that the reweighting of the subtests in favour of DR, VF and LM, may have overestimated their contribution to the sensitivity of the Qmci, minimising the role of the other subtests. However, the overall improved sensitivity of the Qmci over the ABCS 135, in differentiating MCI from NC, suggests that the reweighting and addition of LM, have enhanced the test as a whole. Including only caregivers, attending with subjects, as normal controls, could also have led to bias, as the challenge in diagnosing MCI lies in differentiating MCI from persons with subjective memory problems who have NC. This population accounts for up to 50% of referrals in memory clinics [30], but accounted for <10% of our clinic population.

In summary, this study confirms that reweighting the Qmci subtests and adding LM, improved the ability of the original ABCS 135, to differentiate subjects with MCI and NC. This paper further highlights and describes some attributes of an ideal short cognitive screening test for MCI that can be used in everyday clinical practice. The Qmci incorporates several important cognitive domains, across the spectrum of cognition and its subtests allow discrimination of MCI from both NC and dementia, allowing monitoring of progression. The Qmci also has the advantage of being quick to administer, easily translatable (linguistically and culturally), and of having alternative forms. Other tools, such as the SMMSE and the ABCS 135, are less sensitive and because of their scoring range, are less practical for use clinically. Comparison with other rapid screening tools, such as the MoCA, is now required.

Key points.

• All subtests of the Qmci differentiated MCI from NC and dementia.

• LM is the best performing test, registration the worst.

• LM is the best subtest at distinguishing mild cognitive impairment in the oldest and most educated people.

Conflicts of interest

None declared.

Funding

The Centre for Gerontology and Rehabilitation is funded by Atlantic Philanthropies and the Health Service Executive of Ireland.