Abstract
Background:
Sensitive and specific instruments are required to screen for cognitive impairment (CI) in busy clinical practice. The Montreal Cognitive Assessment (MoCA) is widely validated but few studies compare it to tests designed specifically to detect mild cognitive impairment (MCI).
Objective:
Comparison of two “MCI specific” screens: the Quick Mild Cognitive Impairment screen (Qmci) and MoCA.
Methods:
Patients with subjective memory complaints (SMC; n = 73), MCI (n = 103), or dementia (n = 274), were referred to a university hospital memory clinic and underwent comprehensive assessment. Caregivers, without cognitive symptoms, were recruited as normal controls (n = 101).
Results:
The Qmci was more accurate than the MoCA in differentiating MCI from controls, area under the curve (AUC) of 0.90 versus 0.80, p = 0.009. The Qmci had greater (AUC 0.81), albeit non-significant, accuracy than the MoCA (AUC 0.73) in separating MCI from SMC, p = 0.09. At its recommended cut-off (<62/100), the Qmci had a sensitivity of 90% and specificity of 87% for CI (MCI/dementia). Raising the cut-off to <65 optimized sensitivity (94%), reducing specificity (80%). At <26/30 the MoCA had better sensitivity (96%) but poor specificity (58%). A MoCA cut-off of <24 provided the optimal balance. Median Qmci administration time was 4.5 (±1.3) minutes compared with 9.5 (±2.8) for the MoCA.
Conclusions:
Although both tests distinguish MCI from dementia, the Qmci is particularly accurate in separating MCI from normal cognition and has shorter administration times, suggesting it is more useful in busy hospital clinics. This study reaffirms the high sensitivity of the MoCA but suggests a lower cut-off (<24) in this setting.
Keywords: Cognitive screening, dementia, mild cognitive impairment, Montreal Cognitive Assessment, Quick Mild Cognitive Impairment screen
INTRODUCTION
As society ages, the prevalence of cognitive impairment (CI) is expected to rise [1, 2], resulting in increased numbers of older people presenting with memory complaints. Memory loss is a spectrum from subjective memory complaints (SMC), which is characterised by normal cognition on neuropsychological testing, to mild cognitive impairment (MCI) and dementia. SMC are common, increase with age, and are associated with cognitive decline [3, 4]. MCI represents a heterogeneous group of disorders of memory impairment, characterized by objective and subjective memory loss but preserved function [5, 6], with variable progression to dementia [7]. Different treatment approaches are recommended depending on where patients fall on the spectrum [8]. Thus, sensitive and specific instruments are required to screen for CI, particularly in busy clinical settings.
The Montreal Cognitive Assessment (MoCA) is a well-established cognitive screen, highly sensitive at differentiating MCI from normal cognition and dementia [9] and is widely validated against the most commonly used instrument, the Mini-Mental State Examination (MMSE) [10, 11], in multiple settings [12–14], disorders [15–17] and languages [18–21]. Normative population data are also available [22, 23]. The MoCA overcomes the high ceiling effects and educational bias associated with the MMSE [24], has fewer practice effects and is available in multiple formats [24]. Although the MoCA is increasingly considered the short cognitive instrument of choice, its use as a screen presents some challenges. It is long, taking at least 10 minutes to complete [9], and its subtest scores are criticized for having low accuracy when predicting impairment in their respective cognitive domains [25]. Its specificity at its recommended cut-off (<26) is low, between 35% [12] and 50% [14], lower than that reported in the original validation cohort [9]. Recently, it has been suggested that lowering its cut-off will improve its specificity without adversely affecting its sensitivity [26].
The Quick Mild Cognitive Impairment screen (Qmci), presented in Supplementary Material 1, is a short screening test for CI that was developed as a rapid, valid, and reliable instrument for the early detection and differential diagnosis of MCI and dementia [27, 28]. It correlates with the standardized Alzheimer‘s Disease Assessment Scale-cognitive section, Clinical Dementia Rating scale and the Lawton-Brody activities of daily living scale [29]. Neither the MoCA nor the Qmci are usually compared to short screens designed specifically to detect MCI as well as dementia. Furthermore, little is known about the optimal cut-off scores for either instrument in patients referred to a clinic. Given this, we chose to compare the Qmci and MoCA, two “MCI specific” screening instruments, in a geriatric memory clinic population.
METHODS
Participants
Patients referred for investigation of memory loss were recruited from a university hospital memory clinic in Cork City, Ireland, between March 2012 and December 2014. Alzheimer’s disease and vascular type dementia were classified using the Diagnostic and Statistical Manual of Mental Disorders (4th-edition) [30]. Severity was correlated with the Reisberg FAST scale [31]. Early dementia was defined clinically as noticeable deficits with demanding organizational tasks, e.g., decreased job function (as opposed to ‘prodromal Alzheimer’s disease’, which is synonymous with ‘MCI due to Alzheimer’s disease’ and defined by biomarkers). Mild dementia was defined if assistance in complicated instrumental activities such as handling medications and finances etc. was required. MCI was diagnosed using Petersen’s criteria [32] according to the National Institute on Aging-Alzheimer’s Association workgroup diagnostic guidelines [6]. Frontotemporal dementia (FTD) was diagnosed clinically referencing the Lund-Manchester Criteria [33]. FTD MCI was diagnosed clinically with reference to proposed criteria [34]. Parkinson’s disease dementia (PDD) and MCI were defined by the Movement Disorder Society Guidelines [35, 36], Lewy body dementia (LBD) and MCI using the third report of the LBD Consortium [37]. SMC was defined as subjective non-progressive memory complaints in patients without objective cognitive deficits or functional decline, scoring ‘poor’ or ‘fair’ on a five-point Likert scale in response to the question “how is your memory?” [38]. Normal controls were recruited by convenience sampling from healthy participants, usually caregivers, without cognitive problems accompanying the patients.
Those with active depression (n = 23), aged <45 years (n = 22), declining consent (n = 3), with an unclear diagnosis (n = 21), unable to communicate in English (n = 2), or with resolving delirium in patients recently discharged from hospital (n = 2), were excluded. Depression was excluded clinically and screened with the Geriatric Depression Scale short-form [39] (cut-off ≥7, to optimize specificity [40]). Functional level was measured clinically with the assistance of the Barthel Index [41]. Unless there was co-existing physical disability, all patients diagnosed with SMC or MCI had a normal Barthel Index score of 20/20.
Outcome measures
The Qmci has six subtests, covering five domains: orientation, registration, clock drawing, delayed recall, verbal fluency (VF) (a test of semantic verbal fluency, e.g., naming of animals within one minute) and logical memory (LM) (testing immediate verbal recall of a short story) [27, 28]. Scored out of 100 points, it has a median administration time of 4.24 minutes [28]. The recommended cut-off score for CI (MCI or dementia) is <62 [42]. The MoCA is scored out of 30 points and has seven subtests, covering five cognitive domains: visuospatial/executive function, naming, memory, attention, language, abstraction, delayed recall and orientation [9, 25]. For screening in clinics, where high sensitivity is required, the established MoCA threshold of <26 is suggested [9] although a lower threshold (<24) may have better predictive value [24].
Data collection
Consecutive referrals underwent a comprehensive work-up including history, physical examination, laboratory testing, neuropsychological assessment, and neuroimaging, usually over two sessions, approximately six months apart, to maximize the accuracy of the final diagnosis. Two informant-rated assessments, the 8-item AD8 questionnaire [43, 44] and the Informant Questionnaire on Cognitive Decline in the Elderly-short form [45], were used to inform the diagnosis. Cognitive screening with the Qmci and MoCA was performed in a random counterbalanced order, alternating which of the two tests was scored first to reduce learning or fatigue effects, approximately one hour before consultant review, by two independent trained raters, blind to each other and the final diagnosis. Alternate validated versions of VF and LM were used for the Qmci to reduce learning effects [46]. Normal controls underwent a similar comprehensive review but did not undergo laboratory testing or neuroimaging and few were available for a second evaluation. The study adhered to the tenets of the Declaration of Helsinki. Ethics approval was obtained from the Clinical Research Ethics Committee of the Cork Teaching Hospitals and where possible participants provided written consent; assent was obtained from the relatives or caregivers of individuals who were felt to lack capacity in accordance with current Irish law.
Analysis
Data were analyzed using SPSS 20.0. The Shapiro–Wilk test was used to test normality and found that the majority of data were non-parametric. These were compared using the Mann–Whitney U test. Analysis of covariance (ANCOVA) was used to control the results of analysis for age and education. Accuracy was assessed with receiver operating characteristics (ROC) curves, compared with the Hanley method [47]. Binary logistic regression was used to control ROC curves for the effects of age and education. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for all tests at different cut-off points and by age (≤75 and >75 to balance numbers between groups) and education (<12 and ≥12, mean education in the United Kingdom and Ireland [48]).
RESULTS
In total, 551 participants were included: 73 with SMC, 103 with MCI, 274 with dementia, and 101 normal controls. Of those with MCI, 79% (n = 81) were amnestic type MCI and 21% (n = 22) were non-amnestic; 60% (n = 62) had deficits in a single domain and 35% (n = 36) in multi-domains. Five could not be clearly identified as single or multi-domain. The median age of participants was 76 y (interquartile range, IQR±12) and the majority were female (n = 363, 66%). Patients with dementia (median age of 77±10 years) were significantly older than those with SMC (72±11, p < 0.001), MCI (76±13, p = 0.03) and normal controls (74±14, p < 0.001). The median time in education was 12±4 y. Those with dementia had spent less time in education (11±3 y) compared to those with SMC (12±4 y, p = 0.07), MCI (13±5 y, p < 0.001), or controls (13±4 y, p < 0.001). There was no significant difference in age (p = 0.82) or education (p = 0.16) between those with SMC and controls. Patients with MCI were of a similar age to those with dementia (p = 0.06) but were significantly older than controls (p = 0.01); they had also received more time in education than patients with dementia (p < 0.001) and normal controls (p < 0.001). To control for age and education, ANCOVA was used to test differences between participant groups. This confirmed a statistically significant difference in Qmci scores between all three groups (controls, MCI, and dementia), irrespective of age or education, F(2,333) = 311.96, p < 0.001, partial η2 = 0.65. A similar effect was found for the MoCA: F(2,333) = 190.20, p < 0.001, partial η2 = 0.53. The Qmci screen scores had a “stronger” difference between the three groups compared to the MoCA, based on a higher value of the partial eta-squared (effect size).
The majority of patients with dementia were classified as early to mild stage dementia (n = 201, 73%). Participant characteristics, median test scores according to diagnosis and the prevalence of MCI and dementia subtypes are presented in Table 1. Median administration time for the Qmci was 4.5 (±1.3) minutes, compared to 9.5 (±2.8) for the MoCA.
Table 1.
Baseline characteristics of patients including the distribution of dementia and mild cognitive impairment (MCI) subtypes, median Quick Mild Cognitive Impairment (Qmci) screen and Montreal Cognitive Assessment (MoCA) scores, with inter-quartile range (IQR) values, for normal controls and patients with subjective memory complaints (SMC), MCI, and dementia
| Group | Total | Normal | SMC | MCI | Dementia | Early-Mild | Moderate | Severe |
| Included | Controls | (Total) | ||||||
| Participants | 551 | 101 | 73 | 103 | 274 | 201 | 61 | 12 |
| Age | ||||||||
| (Years) | 76±12 | 74±14 | 72±11 | 76±13 | 77±10 | 77±9 | 79±15 | 77±11 |
| (Median±IQR) | ||||||||
| (Q3-Q1) | (81-69) | (78-64) | (78-67) | (81-68) | (82-72) | (82-73) | (84-69) | (79-68) |
| Education | ||||||||
| (Years) | 12±4 | 13±4 | 12±4 | 13±5 | 11±3 | 11±3 | 10±2 | 13±3 |
| (Median±IQR) | ||||||||
| (Q3-Q1) | (14-10) | (15-11) | (14-10) | (16-11) | (13-10) | (13-10) | (11-9) | (13-10) |
| Gender (% Female) | 66% | 69% | 61% | 66% | 68% | 68% | 70% | 58% |
| Qmci | ||||||||
| (Median±IQR) | 51±29 | 86±13 | 68±9 | 56±12 | 37±23 | 41±14 | 18±17 | 7±6 |
| (Q3-Q1) | ||||||||
| (66-37) | (81-68) | (73-64) | (64-52) | (45-22) | (46-32) | (26-9) | (9-3) | |
| MoCA | ||||||||
| (Median±IQR) | 20±12 | 26±4 | 25±4 | 22±5 | 13±9 | 16±7 | 6±5 | 2±3.5 |
| (Q3-Q1) | ||||||||
| (25-13) | (28-24) | (27-23) | (25-20) | (17-8) | (19-12) | (9-4) | (3.5-0) | |
| Subtype | Alzheimer’s | Vascular | Mixed | Frontotemporal | Lewy Body | Parkinson’s | Other/ | |
| (possible/probable) | disease | unknown | ||||||
| Dementia (%) | 176 (64%) | 40 (15%) | 24 (9%) | 8 (3%) | 12 (4%) | 3 (1%) | 11 (4%) | |
| MCI (%) | 72 (70%) | 14 (13.5%) | 8 (8%) | 5 (5%) | 3 (3%) | 1 (0.5%) | – | |
| SMC | – | – | – | – | – | – | – |
Comparing the accuracy of the tests at differentiating normal controls from MCI showed that the Qmci had significantly greater accuracy, area under the curve (AUC) of 0.90 (95% confidence interval: 0.86–0.94) than the MoCA, AUC of 0.80 (95% confidence interval: 0.74–0.86), p = 0.009. The Qmci was also significantly more accurate than the MoCA in separating normal controls from patients with CI (i.e., MCI and dementia), an AUC of 0.94 versus 0.90 respectively, p = 0.04. In their ability to discriminate SMC from MCI, the Qmci had better (AUC 0.81) accuracy than the MoCA (AUC 0.73), p = 0.09, albeit a non-significant difference. Both instruments had similar, excellent accuracy at differentiating MCI from dementia (AUC of 0.95 versus 0.91 respectively, p = 0.2), and patients with SMC from CI (AUC of 0.97 versus 0.93 respectively, p = 0.23). Both were poor at discriminating normal controls from SMC, (p = 0.28). ROC curves are presented in Fig. 1. Correcting the ROC curves for the effects of age and education showed that the Qmci more accurately differentiated MCI from normal controls (AUC of 0.94; 95% confidence interval: 0.90–0.97) compared with the MoCA (AUC of 0.84; 95% confidence interval: 0.77–0.90), z = 2.76, p = 0.006, see Fig. 2a. The Qmci was also significantly better at separating MCI from dementia (AUC 1.00; 95% confidence interval: 0.998–1.00) than the MoCA (AUC of 0.978; 95% confidence interval: 0.96–0.99), a small but statistically significant difference, z = 2.69, p = 0.007, see Fig. 2b.
Fig.1.
Receiver Operating Characteristic curves demonstrating the accuracy of the Quick Mild Cognitive Impairment (Qmci) screen and Montreal Cognitive Assessment (MoCA) in differentiating (a) mild cognitive impairment (MCI) from normal controls, (b) MCI from subjective memory complaints (SMC), (c) MCI and dementia, (d) normal controls from cognitive impairment (MCI and dementia), (e) SMC from cognitive impairment, and (f) SMC from normal controls.
Fig.2.
Comparison of the Receiver Operating Characteristic curves demonstrating the accuracy of the Quick Mild Cognitive Impairment (Qmci) screen and Montreal Cognitive Assessment (MoCA) in differentiating (a) mild cognitive impairment (MCI) from normal controls and (b) MCI from dementia. (note: data are adjusted for age and education)
The ability of both instruments to separate normal controls from CI was then assessed at different cut-off scores. Patients with SMC were analyzed separately. At their established cut-offs, <62 for the Qmci and <26 for the MoCA, the Qmci had a sensitivity of 90% and specificity of 87% (PPV of 0.96, NPV 0.70) for CI, compared to 96% sensitivity and 58% specificity (PPV of 0.89, NPV 0.80) for the MoCA (<26). At these cut-offs the MoCA had a false positive rate of 11% compared to 4% for the Qmci; the MoCA misclassified 42/101 (42%) of controls as having CI compared to 13/101 (13%) with the Qmci. Increasing the Qmci cut-off to <65 improved the sensitivity (94%) but reduced the specificity (80%). Reducing the MoCA cut-off for CI to <24 yielded an optimum sensitivity and specificity for the test, 89% and 83% respectively, and reduced the false positive rate to 5% . Sensitivity, specificity, PPV, and NPVs are presented in Table 2.
Table 2.
Sensitivity, Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV), with 95% confidence intervals (CI), for different Quick Mild Cognitive Impairment (Qmci) screen and Montreal Cognitive Assessment (MoCA) cut-off scores for cognitive impairment (mild cognitive impairment and dementia), without adjustment for age or education, compared with normal controls
| Cognitive | Sensitivity | Specificity | PPV | NPV | False | False |
| Screen | (95% CI) | (95% CI) | (95% CI) | (95% CI) | Positive | Negative |
| Cut-off score | (95% CI) | (95% CI) | ||||
| Prevalence of cognitive | ||||||
| impairment of 79% | ||||||
| Q mci | ||||||
| <65 | 94% | 80% | 94% | 80% | 6% | 20% |
| (91–96) | (71–87) | (91–96) | (71–87) | (4–7) | (13–29) | |
| <64 | 93% | 82% | 95% | 76% | 5% | 24% |
| (89–95) | (73–89) | (92–97) | (69–84) | (3–8) | (16–33) | |
| <63 | 91% | 85% | 96% | 74% | 4% | 26% |
| (88–94) | (76–91) | (93–97) | (64–81) | (3–7) | (19–36) | |
| <62† | 90% | 87% | 96% | 70% | 4% | 30% |
| (86–93) | (79–93) | (93–98) | (61–78) | (2–7) | (22–39) | |
| <61 | 88% | 92% | 98% | 69% | 2% | 31% |
| (84–91) | (85–96) | (95–99) | (60–76) | (1–5) | (24–40) | |
| <60 | 87% | 93% | 98% | 67% | 2% | 33% |
| (83–90) | (86–97) | (95–99) | (59–75) | (1–5) | (25–41) | |
| <59 | 85% | 96% | 99% | 65% | 1% | 35% |
| (81–89) | (90–99) | (96–100) | (56–72) | (0–4) | (28–44) | |
| <58 | 84% | 98% | 99% | 64% | 1% | 36% |
| (80–88) | (92–100) | (97–100) | (56–71) | (0–3) | (29–44) | |
| <57 | 84% | 100% | 100% | 64% | 0% | 36% |
| (79–87) | (95–100) | (98–100) | (55–71) | (0–2) | (29–45) | |
| <56 | 82% | 100% | 100% | 61% | 0% | 39% |
| (78–86) | (95–100) | (98–100) | (53–69) | (0–2) | (31–47) | |
| <55 | 81% | 100% | 100% | 59% | 0% | 41% |
| (76–85) | (95–100) | (98–100) | (52–67) | (0–2) | (33–48) | |
| MoCA | ||||||
| <26* | 96% | 58% | 89% | 80% | 11% | 20% |
| (93–98) | (48–68) | (85–92) | (68–88) | (8–15) | (12–32) | |
| <24** | 89% | 83% | 95% | 69% | 5% | 31% |
| (86–92) | (74–90) | (92–97) | (60–77) | (3–8) | (23–40) | |
| <23*** | 85% | 86% | 96% | 62% | 4% | 38% |
| (81–88) | (77–92) | (93–97) | (53–70) | (3–7) | (30–47) | |
| <22**** | 78% | 91% | 97% | 54% | 3% | 46% |
| (74–82) | (83–96) | (94–98) | (46–62) | (2–6) | (38–54) | |
†Cut-off for cognitive impairment selected from O’Caoimh et al. [42]. *Cut-off for cognitive impairment selected from Nasreddine et al. [9]. **Cut-off for cognitive impairment selected from Damian et al. [24]. ***Cut-off for cognitive impairment selected from Luis et al. [12]. ****Cut-off for mild cognitive impairment selected from Freitas et al. [26].
Adjusting for age and education showed that the Qmci was most sensitive and specific for CI in patients with less education (<12 years), with a sensitivity and specificity of 99% and 85% for those aged ≤75; 97% and 74% respectively for those aged >75 years. Sensitivity was lowest for younger patients aged ≤75 with ≥12 years in formal education (74%). The MoCA had similar excellent sensitivity (99%) for older patients (>75) with less education (<12) but very low specificity (37.5%). The MoCA had greater sensitivity for those with more education. These values, adjusted for age and education, are presented in Table 3. Reanalysis of the data comparing the ability of the tests at their established cut-offs to differentiate SMC from MCI and dementia showed similar results, see Supplementary Material 2.
Table 3.
Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and false positive and negative values for the Quick Mild Cognitive Impairment (Qmci) screen and Montreal Cognitive Assessment (MoCA), based upon cut-off scores for cognitive impairment (MCI and dementia), according to age and education (CI = 95% Confidence Interval)
| Cognitive | Age | Edu | Median | N = X | Cut-off | Sensitivity | Specificity | PPV | NPV | False | False |
| Screen | Score | score | (95% CI) | (95% CI) | (95% CI) | (95% CI) | Positive | Negative | |||
| (IQR) | (95% CI) | (95% CI) | |||||||||
| Cognitive | |||||||||||
| Impairment | |||||||||||
| Controls | |||||||||||
| Qmci | >75 | <12 | 42 (50-26 = 24) | 146/170 (86%) | 97% | 75% | 96% | 78% | 4% | 22% | |
| 70 (75-63 = 12) | (92–99) | (53–89) | (91–98) | (56–92) | (2–7) | (8–44) | |||||
| >75 | ≥12 | 47 (58-37 = 21) | 59/76 (78%) | <62 | 80% | 94% | 98% | 55% | 2% | 49% | |
| 78 (81-71 = 10) | (68–89) | (69–100) | (89–100) | (36–73) | (0–11) | (27–64) | |||||
| ≤75 | ≥12 | 47 (62-33 = 29) | 73/110 (66%) | 74% | 95% | 97% | 64% | 3% | 36% | ||
| 79 (84-75 = 9) | (63–83) | (80–99) | (87–99) | (50–76) | (1–13) | (24–50) | |||||
| ≤75 | <12 | 41 (48-22 = 26) | 77/97 (79%) | 99% | 85% | 96% | 94% | 4% | 6% | ||
| 74.5 (82-65 = 17) | (92–100) | (61–96) | (89–99) | (71–100) | (1–11) | (0–29) | |||||
| MoCA | >75 | <12 | 14.5 (19-10 = 9) | 146/170 (86%) | 99% | 37.5% | 91% | 82% | 9% | 18% | |
| 25 (26-23 = 3) | (95–100) | (20–59) | (85–94) | (48–97) | (6–15) | (3–52) | |||||
| >75 | ≥12 | 19 (22-13 = 9) | 59/76 (78%) | <26 | 91.5% | 71% | 91.5% | 71% | 8% | 29% | |
| 27 (29-25 = 4) | (81–97) | (44–89) | (81–97) | (44–89) | (3–19) | (11–56) | |||||
| ≤75 | ≥12 | 18 (23-12 = 11) | 73/110 (66%) | 90% | 73% | 87% | 79% | 13% | 21% | ||
| 27 (30-25 = 6) | (81–96) | (56–86) | (77–93) | (62–91) | (7–23) | (9–38) | |||||
| ≤75 | <12 | 16 (20-7 = 13) | 77/97 (79%) | 99% | 50% | 88% | 91% | 12% | 9% | ||
| 25.5 (27-24 = 3) | (92–100) | (28–72) | (79–94) | (57–100) | (6–21) | (1–43) |
DISCUSSION
As populations age more patients are presenting to increasingly busy outpatient clinics, many of which are under-resourced [49], necessitating the use of short instruments to identify MCI and monitor progression to dementia. Prompt diagnosis is particularly important as new management strategies emerge [50, 51]. With multiple instruments available for MCI [52] and dementia [53], choosing one instrument is challenging. This paper explores the accuracy, sensitivity, and specificity of the Qmci and MoCA in their ability to detect CI (MCI and dementia) and differentiate normal controls from SMC, MCI, and dementia.
The results suggest that the Qmci is briefer than the MoCA and particularly accurate in distinguishing MCI from controls. Both instruments had excellent accuracy in separating MCI from dementia, and normal cognition and SMC from MCI and CI. The results reaffirm the high sensitivity of the MoCA but show that the Qmci has excellent sensitivity and specificity. The established cut-off scores did not provide the highest sensitivity and specificity for either instrument. At the widely used cut-off of <26 [9], the MoCA had 96% sensitivity but only 58% specificity for detecting CI compared to 90% and 87% respectively for the Qmci at a cut-off of <62 [42]. The poor specificity of the MoCA at this cut-off is similar to results published elsewhere [12, 15, 17, 24, 26]. Specificity improved when the cut-off was lowered and from this data, the optimal cut-off appears to be that suggested by Damian et al. at <24 [24]. The Qmci was associated with fewer false positive results for CI, 4% at <62, compared to 11% for the MoCA at <26. At this cut-off, a large percentage of controls (42%) screened positive using the MoCA. While high sensitivity is desirable for any screening test, false positive rates of this magnitude may result in large numbers undergoing unnecessary investigation, negating the purpose of screening. Similar results were found for identifying those with SMC. Indeed, neither instrument was accurate in distinguishing SMC from normal controls, which reflects the challenges in defining this condition [54]. Although all patients received interval assessment, this duration may not have been sufficient to see progression in those diagnosed with SMC.
This study also reaffirms that cognitive instruments require adjustment for age and education. The Qmci had low sensitivity in those with more time in formal education (≥12 years) and the MoCA low specificity in those with less time in formal education (<12 years). This is similar to other studies demonstrating that established MoCA cut-offs lack accuracy, particularly specificity, among older adults and those with less time in formal education [22].
The study suggests several potential advantages of the Qmci over the MoCA in this clinic sample. The Qmci is more efficient because it takes half the time to complete compared to the MoCA. The MoCA had significant floor effects (median score of two points in severe cases), making it particularly difficult for those with severe dementia to complete. As the Qmci is scored from 100 points, each subtest provides more information. This is exemplified by the scoring of the Clock. Although requiring more interpretation, the scoring of the Qmci-Clock, provides more detailed information and contributes more to the final test score. The Qmci also incorporates fewer subtests that require normal vision than the MoCA. Visual impairment affects the performance of subjects on cognitive testing, particularly in older adults [55]. Visual tasks account for 27% of the MoCA’s (Visuospatial/Executive and Naming) overall score compared to 15% for the Qmci (clock drawing). This said, both tests can be corrected to account for incomplete data and recently a modified version of the MoCA for the visually impaired has been validated [56]. However, the elimination of subtests that require vision (naming, visuospatial and executive function) reduces the discriminating function of the ‘MoCA-BLIND’, particularly its ability to differentiate MCI from controls [56]. Given that this is the principal advantage of the MoCA, suggests that it is overly weighted towards visual tests, important when assessing older adults [56]. The Qmci, on the other hand, derives its accuracy for identifying MCI from its delayed recall, VF, and LM subtests [28].
The main advantage of the MoCA is its sensitivity, the most important psychometric property for screening instruments [57], particularly in those with higher levels of education. The Qmci however, provided an arguably better balance, particularly at a higher cut-off (<65). The MoCA is less weighted toward language, with 73% of the test requiring verbal skills versus 85% for the Qmci. The MoCA is validated widely, in different languages and clinical settings. Validation of the Qmci in other countries, languages, and cultures is now underway. The Qmci is translated into several languages and is validated in Dutch [58]. Future validation should also include comparison with the MoCA and other screens for MCI in different clinical populations such as the Addenbrooke’s Cognitive Examination-III [59].
This study has a number of limitations. The sample size was small, underpowering the study, potentially causing bias, and limiting it to a non-inferiority validation study. A power calculation, performed a priori, suggested that approximately 300 participants (normal controls and patients with MCI) would be required to show superiority of one instrument over the other in separating MCI from normal controls. An attempt was made to classify patients with a diagnosis and subtype where possible. However, as this was a study in clinical practice, some dementia subtypes may have been misclassified. Although participants were screened in random order and alternative versions of tests were used, learning effects may have occurred. Patients with SMC were diagnosed clinically and no specific screening test such as the MAC-Q was used [60]. This said, a short Likert scale in response to a single question could substitute as a valid screen [38]. However, the type of instrument selected may affect the diagnostic accuracy and it is suggested that age-anchored reference questions provide the most utility [61]. As participants were a homogenous sample (older Irish Caucasians), attending a single center (a university hospital memory clinic), there is the potential that spectrum bias may reduce the external validity. These effects have been found for patients with MCI and are particularly affected by age and education [62]. As presented in Table 3, participants’ age and education were seen to affect the sensitivity and specificity of both the MoCA and Qmci screen. Future validation studies, using age and education specific cut-offs, could be used to minimize this potential source of bias. Finally, the high prevalence of cognitive impairment (68% of the total sample) affects the ability to interpret the accuracy of tests [57]. However, the high prevalence seen and the characteristics of patients with dementia (significantly older than normal controls) reflect clinical practice.
In summary, this is the first study to compare a short cognitive screen, designed specifically to differentiate MCI from normal and dementia with the MoCA in a “real-life” outpatient setting. While the MoCA overcomes many of the difficulties associated with the MMSE, particularly in those with high levels of education [63], the MoCA is long and has suboptimal specificity among older adults attending a memory clinic, particularly at its established cut-off (<26). As older adults represent the majority of patients who have cognitive screening performed for symptoms of memory loss, the Qmci may be a shorter and more accurate alternative, especially when used with a higher cut-off score. The MoCA may be better with a lower cut-off than the established score, particularly in older adults with high levels of education. Further research is required to confirm these findings and compare the Qmci and MoCA with other “MCI specific” instruments and in different clinical settings, particularly in primary care, where the brevity and usability of the Qmci is likely to be of most benefit.
Supplementary Material
ACKNOWLEDGMENTS
The Centre for Gerontology and Rehabilitation is supported by Atlantic Philanthropies, the Health Service Executive of Ireland, the Health Research Board of Ireland and the Irish Hospice Foundation. The authors would like to thank Dr. Brian Daly and the nurses in the assessment and treatment center, St. Finbarr’s Hospital for their assistance.
Authors’ disclosures available online (http://www.j-alz.com/manuscript-disclosures/15-0881r2).
Appendix
The supplementary material is available in the electronic version of this article: http://dx.doi.org/10.3233/JAD-150881.
REFERENCES
- [1]. Plassman BL, Langa KM, Fisher GG, Heeringa SG, Weir DR, Ofstedald MB, Burke JR, Hurd MD, Potter GG, Rodgers WL, Steffens DC (2007) Prevalence of dementia in the United States: The Aging, Demographics and Memory Study. Neuroepidemiology 29, 125–132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2]. Plassman BL, Langa KM, Fisher GG, Heeringa SG, Weir DR, Ofstedal MB, Burke JR, Hurd MD, Potter GG, Rodgers WL, Steffens DC, McArdle JJ, Willis RJ, Wallace RB (2008) Prevalence of cognitive impairment without dementia in the United States. Ann Intern Med 148, 427–434. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3]. Wang L, van Belle G, Crane PK, Kukull WA, Bowen JD, McCormick WC, Larson EB (2004) Subjective memory deterioration and future dementia in people aged 65 and older. J Am Geriatr Soc 52, 2045–2051. [DOI] [PubMed] [Google Scholar]
- [4]. Reisberg B, Shulman MB, Torossian C, Leng L, Zhu W (2010) Outcome over seven years of healthy adults with and without subjective cognitive impairment. Alzheimers Dement 6, 11–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5]. Petersen RC (2004) Mild cognitive impairment as a diagnostic entity. J Intern Med 256, 183–194. [DOI] [PubMed] [Google Scholar]
- [6]. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A, Holtzman DM, Jagust WJ, Petersen RC, Snyder PJ (2011) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 270–279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7]. Mitchell AJ, Shiri-Feshki M (2009) Rate of progression of mild cognitive impairment to dementia— meta-analysis of 41 robust inception cohort studies. Acta Psychiatr Scand 119, 252–265. [DOI] [PubMed] [Google Scholar]
- [8]. Tricco AC, Soobiah C, Berliner S, Ho JM, Ng CH, Ashoor HM, Chen MH, Hemmelgarn B, Straus SE (2013) Efficacy and safety of cognitive enhancers for patients with mild cognitive impairment: A systematic review and meta-analysis. CMAJ 185, 1393–1401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9]. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, Chertkow H (2005) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 695–699. [DOI] [PubMed] [Google Scholar]
- [10]. Folstein MF, Folstein SE, McHugh PR (1975) ‘Mini-mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12, 189–198. [DOI] [PubMed] [Google Scholar]
- [11]. Shulman KI, Herrmann N, Brodaty H, Chiu H, Lawlor B, Ritchie K, Scanlan JM (2006) IPA survey of brief cognitive screening instruments. Int Psychogeriatrics 18, 281–294. [DOI] [PubMed] [Google Scholar]
- [12]. Luis CA, Keegan AP, Mullan M (2009) Cross validation of the Montreal Cognitive Assessment in community dwelling older adults residing in the Southeastern US. Int J Geriatr Psychiatry 24, 197–201. [DOI] [PubMed] [Google Scholar]
- [13]. Olson RA, Iverson GL, Carolan H, Parkinson M, Brooks BL, McKenzie M (2011) Prospective comparison of two cognitive screening tests: Diagnostic accuracy and correlation with community integration and quality of life. J Neurooncol 105, 337–344. [DOI] [PubMed] [Google Scholar]
- [14]. Smith T, Gildeh N, Holmes C (2007) The Montreal Cognitive Assessment: Validity and utility in a memory clinic setting. Can J Psychiatry 52, 329–332. [DOI] [PubMed] [Google Scholar]
- [15]. Hoops S, Nazem S, Siderowf AD, Duda JE, Xie SX, Stern MB, Weintraub D (2009) Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology 73, 1738–1745. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [16]. Videnovic A, Bernard B, Fan W, Jaglin J, Leurgans S, Shannon KM (2010) The Montreal Cognitive Assessment as a screening tool for cognitive dysfunction in Huntington’s disease. Mov Disord 25, 401–404. [DOI] [PubMed] [Google Scholar]
- [17]. Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C, Canaple S, Petitnicolas G (2011) Is the Montreal Cognitive Assessment superior to the Mini-Mental State Examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke 42, 1712–1716. [DOI] [PubMed] [Google Scholar]
- [18]. Thissen AJ, van Bergen F, de Jonghe JF (2010) Applicability and validity of the Dutch version of the Montreal Cognitive Assessment (MoCA-d) in diagnosing MCI. Gerontol Geriatr 41, 231–240. [DOI] [PubMed] [Google Scholar]
- [19]. Selekler K, Cangoz B, Uluc S (2010) Power of discrimination of Montreal Cognitive Assessment (MoCA) scale in Turkish patients with mild cognitive impairment and Alzheimer’s disease. Turkish J Geriatr 13, 166–171. [Google Scholar]
- [20]. Lee JY, Dong Woo Lee, Cho SJ, Na DL, Jeon HJ, Kim SK, Lee YR, Youn JH, Kwon M, Lee JH, Cho MJ (2008) Brief screening for mild cognitive impairment in elderly outpatient clinic: Validation of the Korean version of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol 21, 104–110. [DOI] [PubMed] [Google Scholar]
- [21]. Fujiwara Y, Suzuki H, Yasunaga M, Sugiyama M, Ijuin M, Sakuma N, Inagaki H, Iwasa H, Ura C, Yatomi N, Ishii K (2010) Brief screening tool for mild cognitive impairment in older Japanese: Validation of the Japanese version of the Montreal Cognitive Assessment. Geriatr Gerontol Int 10, 225–232. [DOI] [PubMed] [Google Scholar]
- [22]. Rossetti HC, Lacritz LH, Cullum CM, Weiner MF (2011) Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology 77, 1272–1275. [DOI] [PubMed] [Google Scholar]
- [23]. Kenny RA, Coen RF, Frewen J, Donoghue OA, Cronin H, Savva GM (2013) Normative values of cognitive and physical function in older adults: Findings from The Irish Longitudinal Study on Ageing. J Am Geriatr Soc 61, S279–S290. [DOI] [PubMed] [Google Scholar]
- [24]. Damian A, Jacobsen S, Hentz J, Belden C, Shill H, Sabbagh M, Caviness J, Adler C (2011) The Montreal Cognitive Assessment and the Mini-Mental State Examination as screening instruments for cognitive impairment: Item analyses and threshold scores. Dement Geriatr Cogn Disord 31, 126–131. [DOI] [PubMed] [Google Scholar]
- [25]. Moafmashhadi P, Koski L (2013) Limitations for Interpreting Failure on Individual Subtests of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol 26, 19–28. [DOI] [PubMed] [Google Scholar]
- [26]. Freitas S, Simões MR, Alves L, Santana I (2013) Montreal cognitive assessment: Validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord 27, 37–43. [DOI] [PubMed] [Google Scholar]
- [27]. O’Caoimh R, Gao Y, McGlade C, Healy L, Gallagher P, Timmons S, Molloy DW (2012) Comparison of the Quick Mild Cognitive Impairment (Qmci) screen and the SMMSE in screening for mild cognitive impairment. Age Ageing 41, 624–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [28]. O’Caoimh R, Gao Y, Gallagher P, Eustace J, McGlade C, Molloy DW (2013) Which part of the Quick mild cognitive impairment screen (Qmci) discriminates between normal cognition, mild cognitive impairment and dementia?. Age Ageing 42, 324–330. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29]. O’Caoimh R, Svendrovski A, Johnston B, Gao Y, McGlade C, Timmons S, Eustace J, Guyatt G, Molloy DW (2014) Comparison of the Quick mild cognitive impairment screen (Qmci) to the Standardised Alzheimer‘s Disease Assessment Scale-cognitive section (SADAS-cog) in clinical trials. J Clin Epidemiol 67, 87–92. [DOI] [PubMed] [Google Scholar]
- [30]. American PsychiatricAssociation (1994) Diagnostic and Statistical Manual of Mental Disorders, 4th edition American Psychiatric Association: Washington, DC. [Google Scholar]
- [31]. Reisberg B (1988) Functional Assessment Staging (FAST). Psychopharmacol Bull 24, 653–659. [PubMed] [Google Scholar]
- [32]. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E (1999) Mild cognitive impairment: Clinical characterization an outcome. Arch Neurol 56, 303–308. [DOI] [PubMed] [Google Scholar]
- [33]. Brun A, Englund B, Gustafson L, Passant U, Mann DMA, Neary D, Snowden JS (1994) Clinical and neuropathological criteria for frontotemporal dementia. The Lund and Manchester Groups. J Neurol Neurosurg Psychiatry 57, 416–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [34]. de Mendonça A, Ribeiro F, Guerreiro M, Garcia C (2004) Frontotemporal mild cognitive impairment. J Alzheimers Dis 6, 1–9. [DOI] [PubMed] [Google Scholar]
- [35]. Emre M, Aarsland D, Brown R, Burn DJ, Duyckaerts C, Mizuno Y, Broe GA, Cummings J, Dickson DW, Gauthier S, Goldman J (2007) Clinical diagnostic criteria for dementia associated with Parkinson’s disease. Mov Disord 22, 1689–1707. [DOI] [PubMed] [Google Scholar]
- [36]. Litvan I, Goldman JG, Tröster AI, Schmand BA, Weintraub D, Petersen RC, Mollenhauer B, Adler CH, Marder K, Williams-Gray CH, Aarsland D (2012) Diagnostic criteria for mild cognitive impairment in Parkinson’s disease: Movement Disorder Society Task Force guidelines. Mov Disord 27, 349–356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [37]. McKeith IG, Dickson DW, Lowe J, Emre M, O’brien JT, Feldman H, Cummings J, Duda JE, Lippa C, Perry EK, Aarsland D (2005) Consortium on DLB. Diagnosis and management of dementia with Lewy bodies: Third report of the DLB Consortium. Neurology 65, 1863–1872. [DOI] [PubMed] [Google Scholar]
- [38]. Paradise MB, Glozier NS, Naismith SL, Davenport TA, Hickie IB (2011) Subjective memory complaints, vascular risk factors and psychological distress in the middle-aged: A cross-sectional study. BMC Psychiatr 11, 108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [39]. Yesavage JA (1988) Geriatric depression scale. Psychopharmacol Bull 24, 709–711. [PubMed] [Google Scholar]
- [40]. Marc LG, Raue PJ, Bruce ML (2008) Screening performance of the Geriatric Depression Scale (GDS-15) in a diverse elderly home care population. Am J Geriatr Psychiatry 16, 914–921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [41]. Collin C, Wade D, Davies S, Horne V (1988) The Barthel ADL Index: A standard measure of disability?. Int Disabil Stud 10, 64–67. [DOI] [PubMed] [Google Scholar]
- [42]. O’Caoimh R, Gao Y, Gallagher P, Eustace J, Molloy W (2014) Cognitive screening tests need to be adjusted for age and education in patients presenting with symptomatic memory loss. Irish J Med Sc 183, 314. [Google Scholar]
- [43]. Galvin JE, Rose CM, Powlishta KK, Coats MA, Muich SJ, Grant E, Miller JP, Storandt M, Morris JC (2005) The AD8. A brief informant interview to detect dementia. Neurology 65, 559–564. [DOI] [PubMed] [Google Scholar]
- [44]. Razavi M, Tolea MI, Margrett J, Martin P, Oakland A, Tscholl DW, Ghods S, Mina M, Galvin JE (2014) Comparison of 2 informant questionnaire screening tools for dementia and mild cognitive impairment: AD8 and IQCODE. Alzheimer Dis Assoc Disord 28, 156–161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [45]. Jorm AF (1994) A short form of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): Development and cross-validation. Psychol Med 24, 145–153. [DOI] [PubMed] [Google Scholar]
- [46]. Cunje A, Molloy DW, Standish TI, Lewis DL (2007) Alternative forms of logical memory and verbal fluency tasks for repeated testing in early cognitive changes. Int Psychogeriatrics 19, 65–65. [DOI] [PubMed] [Google Scholar]
- [47]. Hanley JA, McNeil BJ (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148, 839–843. [DOI] [PubMed] [Google Scholar]
- [48]. United Nations Development Programme (2015) Mean years of schooling (of adults) (years). Human Development Records http://hdr.undp.org/en/content/mean-years-schooling-adults-years.
- [49]. Cahill S, Pierce M, Moore V (2014) A national survey of memory clinics in the Republic of Ireland. Int Psychogeriatrics 26, 605–613. [DOI] [PubMed] [Google Scholar]
- [50]. O’Caoimh R, Sato S, Wall J, Igras E, Foley MJ, Timmons S, Molloy DW (2015) Potential for a “Memory Gym” intervention to delay conversion of mild cognitive impairment to dementia. J Am Med Dir Assoc 16, 998–999. [DOI] [PubMed] [Google Scholar]
- [51]. Fiatarone Singh MA, Gates N, Saigal N, Wilson GC, Meiklejohn J, Brodaty H, Wen W, Singh N, Baune BT, Suo C, Baker MK (2014) The Study of Mental and Resistance Training (SMART) study— resistance training and/or cognitive training in mild cognitive impairment: A randomized, double-blind, double-sham controlled trial. J Am Med Dir Assoc 15, 873–880. [DOI] [PubMed] [Google Scholar]
- [52]. Lonie JA, Tierney KM, Ebmeier KP (2009) Screening for mild cognitive impairment: A systematic review. Int J Geriatr Psychiatry 24, 902–915. [DOI] [PubMed] [Google Scholar]
- [53]. Cullen B, O’Neill B, Evans JJ, Coen RF, Lawlor BA (2007) A review of screening tests for cognitive impairment. J Neurol Neurosurg Psychiatry 78, 790–799. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [54]. Mitchell AJ (2008) Is it time to separate subjective cognitive complaints from the diagnosis of mild cognitive impairment?. Age Ageing 37, 497–499. [DOI] [PubMed] [Google Scholar]
- [55]. Killen A, Firbank MJ, Collerton D, Clarke M, Jefferis JM, Taylor JP, McKeith IG, Mosimann UP (2013) The assessment of cognition in visually impaired older adults. Age Ageing 42, 98–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [56]. Wittich W, Phillips N, Nasreddine ZS, Chertkow H (2010) Sensitivity and specificity of the Montreal Cognitive Assessment modified for individuals who are visually impaired. J Vis Impair Blind 104, 360–368. [Google Scholar]
- [57]. Alberg AJ, Park JW, Hager BW, Brock MV, Diener-West M (2004) The use of “overall accuracy” to evaluate the validity of screening or diagnostic tests. J Gen Intern Med 19, 460–465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [58]. Bunt S, O’Caoimh R, Krijnen WP, Molloy DW, Goodijk GP, van der Schans CP, Hobbelen JSM (2015) Validation of the Dutch version of the quick mild cognitive impairment screen (Qmci-D). BMC Geriat 15, 115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [59]. Larner AJ, Mitchell AJ (2014) A meta-analysis of the accuracy of the Addenbrooke’s Cognitive Examination (ACE) and the Addenbrooke’s Cognitive Examination-Revised (ACE-R) in the detection of dementia. Int Psychogeriatr 26, 555–563. [DOI] [PubMed] [Google Scholar]
- [60]. Crook TH, Feher EP, Larrabee GJ (1992) Assessment of memory complaint in age-associated memory impairment: The MACQ. Int Psychogeriatr 4, 165–176. [DOI] [PubMed] [Google Scholar]
- [61]. Tandetnik C, Farrell MT, Cary MS, Cines S, Emrani S, Karlawish J, Cosentino S (2015) Ascertaining subjective cognitive decline: A comparison of approaches and evidence for using an age-anchored reference group.S1, S. J Alzheimers Dis 48, 43–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [62]. Chopard G, Puyraveau M, Binetruy M, Meyer A, Vandel P, Magnin E, Berger E, Galmiche J, Mauny F (2015) spectrumeffect and bias in the screening test performance for amnestic mild cognitive impairment: What are the clinical implications?. J Alzheimers Dis 48, 385–393. [DOI] [PubMed] [Google Scholar]
- [63]. Crum RM, Anthony JC, Bassett SS, Folstein MF (1993) Population-based norms for the Mini-Mental State Examination by age and educational level. JAMA 269, 2386–2391. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
The supplementary material is available in the electronic version of this article: http://dx.doi.org/10.3233/JAD-150881.