Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 16.
Published in final edited form as: Clin Neuropsychol. 2015 Sep 16;29(6):824–835. doi: 10.1080/13854046.2015.1087596

Longitudinal change in performance on the Montreal Cognitive Assessment in older adults

Sarah A Cooley a, Jodi M Heaps a,b, Jacob D Bolzenius a, Lauren E Salminen a, Laurie M Baker a, Staci E Scott a, Robert H Paul a,b
PMCID: PMC4644436  NIHMSID: NIHMS727892  PMID: 26373627

Abstract

Objective

The Montreal Cognitive Assessment (MoCA) is a brief screening measure commonly used to determine cognitive status among older adults. Despite the popularity of the MoCA, there has been little research into how performance on the MoCA changes over time in healthy older adults.

Methods

The present study examined a sample of older adults (n = 53) recruited for a longitudinal study of healthy aging. Change in total MoCA score at three time-points (baseline, 12 months, and 48 months) and scores from the Repeatable Battery for the Assessment of Neuropsychological Status at five time-points (RBANS; baseline 12 months, 24 months, 36 months, 48 months) were assessed using repeated measures analyses.

Results

Total MoCA score significantly increased across time, particularly between the first and second administrations. Scores did not significantly differ between the second (12 month) and third (48 month) administrations. When grouped by baseline performance, individuals who scored low at baseline significantly improved performance at 12 month testing, but had little change between 12 month and 48 month testing. Conversely, individuals who scored high at baseline did not significantly change between baseline and 12 month testing, but improved between 12 month and 48 month testing. RBANS scores did not significantly change over time.

Conclusions

These results suggest that the MoCA may be susceptible to practice effects, particularly between the first and second administrations. These practice effects should be taken into consideration when repeatedly employing the MoCA to screen for cognitive status in healthy older adults.

Keywords: Practice effects, Aging, Neuropsychological assessment, MoCA

The Montreal Cognitive Assessment (MoCA) is a brief screening measure to determine cognitive status (Nasreddine et al., 2005). Developed in 2005, the MoCA was introduced as a more sensitive alternative to the Mini-Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975) in detecting the presence of mild cognitive impairment (MCI) and mild Alzheimer’s disease (90% and 100% sensitivity, respectively; Nasreddine et al., 2005). Since the publication of the original validation study, other studies have cross-validated the MoCA as a brief measure with substantial sensitivity to MCI and mild Alzheimer’s disease (Freitas, Simões, Alves, & Santana, 2013; Luis, Keegan, & Mullan, 2009). As a result, the MoCA has been increasingly utilized as an effective screening measure for detecting MCI in both clinical and research settings.

The MoCA has additional utility as a measure of cognitive decline in populations independent of Alzheimer’s disease. Studies have demonstrated the sensitivity of the MoCA to detect cognitive impairment in Parkinson’s disease (Hoops et al., 2009), cardiovascular disease and stroke (McLennan, Mathias, Brennan, & Stewart, 2011; Pendlebury, Cuthbertson, Welch, Mehta, & Rothwell, 2010), REM sleep behavior disorder (Gagnon, Postuma, Joncas, Desjardins, & Latreille, 2010), and brain metastases (Olson, Chhanabhai, & McKenzie, 2008).

A recent meta-analysis of practice effects on a range of cognitive tests concluded that practice effects are apparent, particularly in the domains of visual memory, attention, working memory, executive functioning and processing speed, and that greater practice effects are observed for shorter test-retest intervals and tests without alternate forms (Calamia, Markon & Tranel, 2012). Previous studies have concluded that screening instruments, such as the MMSE, are also highly prone to practice effects in healthy older adults and in those diagnosed with dementia at short retest intervals of ten minutes to 1.5 weeks (Galasko, Abramson, Corey-Bloom, & Thal, 1993; Jacqmin-Gadda, Fabrigoule, Commenges, & Dartigues, 1997) and longer retest intervals of three months (Helkala et al., 2002). These practice effects tend to occur between the first and second testing sessions and are likely due to the small number of items on a screening measure (Falleti, Maruff, Collie, & Darby, 2006). Similar practice effects have been observed in two subtests of a more comprehensive screening measure, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph, 1998) over a test-retest interval of two weeks (Dong, Thompson, Tan, Lim, Pang & Chen, 2013) although the change in total RBANS score was not significant. Collectively, these studies indicate that repeated exposure to brief screening measures can result in reduced sensitivity to detect cognitive changes, particularly from the first to second test exposure. Despite being developed as a more sensitive alternative to the MMSE, it is unclear whether the MoCA is similarly prone to these practice effects.

In the past few years the MoCA has emerged as a preferred screening measure of cognitive status in clinical and research settings. However, few studies have explicitly examined the utility of this measure longitudinally. Although two relatively recent studies have concluded that the use of alternate forms of the MoCA over short intervals of time (60 minutes to one month) may eliminate possible practice effects (Costa et al., 2012; Chertkow, Nasreddine, Johns, Phillips, & McHenry, 2011), little research has examined longer test-retest intervals and practice effects on the original MoCA. One recent study that examined change on several neuropsychological tests, including the MoCA, over one 12-month interval identified no change in scores in either Huntington’s disease patients or healthy controls (Toh et al., 2014). However, this study did not include further repeated administrations. These results are vital to support the utility of the MoCA as an accurate measure of cognitive change and decline in longitudinal research or clinical settings.

In the present study we examined changes in performance on the MoCA over a four-year period in healthy older adults. Total MoCA scores were analyzed to reveal differences in performance over time. We hypothesized that MoCA scores would significantly increase from the first testing session to the second, with minimal change thereafter. Results from the total MoCA score analysis were compared to longitudinal performance on a more comprehensive screening measure, the RBANS, to compare changes across time between the two tests.

Methods

Participants

The current study was conducted at the University of Missouri – Saint Louis. Participants were recruited from advertisements in the local community and through the Research Participant Registry of the Washington University in Saint Louis Institute of Clinical and Translational Sciences.

Fifty-three individuals were included in the present study. Participants were drawn from a longitudinal study of healthy aging and were over the age of 50. Both male and female English-speaking individuals were included. Individuals with neurological conditions (e.g. current diagnosis of stroke, dementia, Parkinson’s disease, etc.), diabetes requiring medication (not diet-controlled), history of drug or alcohol abuse, a major psychiatric condition (e.g. schizophrenia, bipolar disorder, untreated anxiety or depression), head injury with loss of consciousness >5 minutes, or other conditions that might affect cognition (e.g. thyroid disease, HIV, epilepsy, multiple sclerosis, or cancer within the last 10 years) were excluded from the study.

Additionally, all participants included in the current study indicated normal-to-mild levels of depression, anxiety and stress around the time of testing on a mood measure (Depression Anxiety Stress Scale; Lovibond & Lovibond, 1995).

All data were collected in compliance with regulations of the local Institutional Review Board (IRB). Each participant provided informed consent and was compensated for participation in the study.

Montreal Cognitive Assessment

The MoCA consists of 13 tasks that cover multiple cognitive domains. These domains include visuospatial (visuoconstruction), naming, learning and memory, attention, language, abstraction, delayed recall and orientation. Scores for tasks in each domain were summed for a total possible score of 30 points. It should be noted that the learning/immediate memory task is not included in the total score.

The original cutoff score of 26 total points on the MoCA, identified as highly sensitive to the detection of cognitive impairment (Nasreddine et al., 2005), was utilized to divide participants into groups based on baseline performance. In the current study, individuals scoring less than 26 total points were classified as “low-scoring” individuals, whereas individuals scoring 26 points or higher were classified as “high-scoring.” In a second analysis, a more stringent cutoff score (≤ 23) was employed to establish groups.

Repeatable Battery for the Assessment of Neuropsychological Status

The RBANS is a screening measure to determine cognitive status (Randolph, 1998). It has been widely used as a measure of MCI and AD and has shown to be sensitive to age-related cognitive decline (Duff, Humphreys-Clark, O’Bryant, Mold, Schiffer, & Sutker, 2008; Duff, Hobson, Beglinger, & O’Bryant, 2010; Randolph, Tierney, Mohr, & Chase, 1998). The RBANS consists of twelve subtests that measure five cognitive domains including visuospatial, language, immediate memory, delayed memory and attention. Scores for each domain were summed for five index scores, and the five index scores were further summed for a total score that was age-standardized (normal M = 100, SD = 15). Alternate forms of the RBANS were utilized in the current study, with Form A given at baseline and 48 month testing, Form B at 12 month testing, Form C at 24 month testing, and Form D at 36 month testing.

Neuropsychological Testing

The MoCA was administered as part of a larger neuropsychological battery at the baseline, 12 month (12M), and 48 month (48M) neuropsychological testing sessions, while the RBANS was administered as part of the larger neuropsychological test battery at all neuropsychological testing sessions. The MoCA was not included in the neuropsychological battery at 24 (24M) and 36 month (36M) testing. Only participants with MoCA scores for all three time points were included in the present study.

The Reading subtest from the Wide Range Achievement Test 4 (WRAT4; Wilkinson & Robertson, 2006) was included within the larger neuropsychological test battery. Baseline performance on this test was analyzed as a measure of premorbid IQ.

Statistical Analyses

All analyses were completed using IBM SPSS Statistics software v. 21 (IBM corp., Armonk, NY). Demographic characteristics (age, sex, years of education and ethnicity) and premorbid IQ were assessed in the current sample. These demographic variables were also compared between individuals with data at all time-points and included in the current study sample (“completers”) and those missing data at one or more time-points (“non-completers”). A general linear model (GLM) repeated measures analysis was completed to assess differences in performance on the MoCA over three time-points: baseline, 12M, and 48M. Another repeated measures analysis examined differences in total MoCA scores over time using baseline MoCA score as a between-subjects grouping variable. The trajectory of total MoCA score for individuals scoring below the conventional MCI cut-off of 26 points (“low-scoring”) was compared to those scoring ≥ 26 (“high-scoring”) to examine whether low scoring individuals at baseline are a greater influence on changes over time (Nasreddine et al., 2005). A secondary repeated measures analysis was conducted utilizing a more stringent MoCA cutoff (≤ 23) as the grouping variable.

Additional repeated measures analyses were completed for RBANS standardized scores at all five time-points, with a focus on the baseline, second (12M) and third (24M) administrations, and RBANS index scores. The Šidák correction was used to adjust p-values and control for multiple comparisons. An adjusted p-value of < .05 was determined to be significant for all analyses.

Test-retest stability coefficients for the MoCA and RBANS were assessed by Pearson’s correlations, and practice-corrected reliable change indices (RCI) were calculated using the method described by Chelune et al. (1993). A measure of internal consistency was calculated using Cronbach’s alpha for the total MoCA and RBANS scores at each time point. Individuals who completed all assessments across the duration of the study were classified as “completers” (N = 53) whereas all others were classified as non-completers (N = 58).

Results

Sample Characteristics

Individuals included in the current study ranged from 51 – 85 years of age (M = 64.02, SD = 7.65). The sample was highly educated (M = 15.70 years, SD = 2.53), included 23 males (43.4%) and 30 females (56.6%), and primarily consisted of Caucasian individuals (43 Caucasian, 7 African American, and 3 Hispanic/Latino participants). There were no significant differences in age, years of education, sex or ethnicity in completers (N = 53) versus non-completers (N = 68; p’s > .05). Additionally, premorbid IQ did not significantly differ between completers (M = 105.62, SD = 10.78) and non-completers (M = 106.00, SD = 9.38; p > .05).

At baseline, approximately 51% of participants (N = 27) fell below the recommended cutoff score for MCI of < 26 on the MoCA (range = 22-30; M = 25.67, SD = 2.30). This number decreased to 25% (N = 13) at 12M testing and 21% (N = 11) at 48M testing. Baseline total MoCA score did not significantly differ between completers and non-completers (p > .05). However, baseline total RBANS score was significantly higher in completers (M = 103.42, SD = 10.99) compared to non-completers (M = 96.40, SD = 13.58; p = .004), though scores for both groups were in the average range. An examination of RBANS domain scores revealed significantly higher scores for completers versus non-completers in immediate memory (completers M = 103.7, SD = 11.2; non-completers M = 97.8, SD = 14.1; p = .02), delayed memory (completers M = 101.6, SD = 11.0; non-completers M = 94.7, SD = 16.5; p = .01) and attention (completers M = 109.2, SD = 12.9; non-completers M = 102.6, SD = 13.8; p = .01) index scores.

Test-retest reliability coefficients and RCI 90% confidence intervals are listed in Table 1. Cronbach’s alpha internal consistency analyses revealed moderate and acceptable internal consistency on the MoCA (baseline α = .62; 12M α = .60; 48M α = .64), while the RBANS analyses revealed good internal consistency (baseline α = .78; 12M α = .77; 24M α = .79; 36M α = .70; 48M α = .78) (Robinson, Shaver & Wrightsman, 1991; Schmitt, 1996; Sekaran, 2005).

Table 1.

Repeated Measures and Reliability Statistics

Measure Mdiff (SD) Effect size
d 		r S.E.diff RCI C.I. (90%)
Total MoCA score
  Baseline - 12 month 0.93(2.6)* 0.45 .33* 2.67 −3.5, +5.3
  12 month - 48 month 0.55(2.2) 0.24 .43** 2.24 −3.1, +4.2
  Baseline - 48 month 1.47(2.2)** 0.72 .48** 2.35 −2.4, +5.3
Total RBANS score
  Baseline - 12 month 0.47(8.5) 0.03 .75** 8.61 −13.7, +14.6
  12 month - 24 month −2.38(7.2) −0.20 .83** 6.80 −13.6, +8.8
  24 month - 36 month 1.41(9.8) 0.12 .64** 10.58 −16.0, +18.8
  36 month - 48 month 2.21(10.8) 0.19 .60** 9.32 −13.1, +17.5
  Baseline - 48 month 1.71(8.4) 0.13 .78** 8.07 −11.6, +15.0
Open in a new tab

Note. MoCA = Montreal cognitive assessment; RBANS = Repeatable battery for the assessment of neuropsychological status; Mdiff = difference in mean scores; d = Cohen's d; r = test-retest correlation; S.E.diff = standard error of difference; RCI C.I. = reliable change index confidence interval

*

p < .05

**

p < .01

Repeated Measures Analyses

Total MoCA score

Performance on the MoCA significantly improved over the three time-points [F (2, 104) = 10.74, p < .001, η2 = .17]. The average MoCA score increased from below the conventional < 26 cutoff at baseline (M = 25.6, SD = 2.30) to above the cutoff at 12M testing (M = 26.6, SD = 2.10) and 48M testing (M = 27.1, SD = 2.04).

Pairwise comparisons identified significant differences between baseline and 12M MoCA scores (p = 0.03; Cohen’s d = 0.45) and baseline and 48M MoCA scores (p < .001; d = 0.69). Performance on the MoCA did not significantly differ between 12M and 48M testing (p = 0.22; d = 0.24; see Table 1).

High-scoring versus low-scoring individuals (MoCA cutoff score < 26)

Twenty-seven individuals scored below 26 points on the MoCA at baseline, the conventional cut-off for MCI, and 26 individuals scored ≥ 26 (see Table 2). In addition to the significant main effect of time, the interaction between group at baseline based on total MoCA score and time was significant [F (2, 102) = 19.13, p < .001, η2 = .27]. Individuals in the lower-scoring group at baseline exhibited significantly increased total MoCA score from baseline to 12M (p < .001; d = 2.03), with little improvement from 12M to 48M (p = .99; d = 0.04). Conversely, those scoring ≥ 26 at baseline scored slightly, but non-significantly, lower at 12M (p = .23; d = 0.38) and significantly higher from 12M to 48M (p = .03; d = 0.47). Group trajectories are depicted in Figure 1.

Table 2.

Montreal Cognitive Assessment (MoCA) Repeated Measures Analysis Results by Baseline Group

Baseline 12 Month 48 Month
Total MoCA Score at
Baseline 		n Mean SD Mean SD Mean SD p
≥ 26 26 27.7 1.2 26.9 2.6 28.0 1.8 <.001*
< 26 27 23.7 1.0 26.3 1.5 26.3 2.0
> 23 44 26.3 1.9 26.7 2.2 27.2 2.1 <.001*
≤ 23 9 22.4 0.5 25.9 1.5 26.7 1.8
Open in a new tab

Note: SD = standard deviation

*

p< 0.01

Figure 1.

Figure 1

Open in a new tab

Mean total Montreal Cognitive Assessment (MoCA) score over time with participants grouped by baseline total MoCA score (≥ 26 versus <26). Error bars represent standard deviation from the mean.

High-scoring versus low-scoring individuals (MoCA cutoff score ≤ 23)

Utilizing the more stringent MoCA cutoff score of ≤ 23, 9 individuals were classified as “low-scoring”, while 44 individuals were classified as “high-scoring” at baseline. There was a significant interaction between time and baseline group (F(2, 102) = 11.02, p < .001, η2 = .18). Individuals scoring ≤ 23 demonstrated a significant increase from baseline to 12M (p = .001; d = 3.0), and changed little between 12M and 48M testing (p = .66; d = 0.48), while individuals scoring > 23 changed little from baseline to 12M (p = .60; d = 0.19) or 12M to 48M testing (p = .38; d = 0.23).

RBANS

Scores on the RBANS did not significantly differ over time [F (4, 180) = 1.32, p = 0.27, η2 = .04].

Discussion

The purpose of the present study was to examine longitudinal performance on the MoCA in healthy older adults. The results provide important insight into the repeated use of the MoCA as a screening measure of cognitive status. Our results indicate that healthy older adults significantly improve their performance on the MoCA from baseline to subsequent testing, with the largest increase occurring between the first and second administration, a retest interval of one year. After this initial increase, scores did not significantly differ from 12M to 48M testing sessions. Test-retest reliability for the MoCA over time was, although significant, generally low-moderate (r = .33-.48). These test-retest reliability coefficients indicate that scores on the MoCA were not stable over time, particularly between the first and second administration. Internal consistency of the MoCA at each time point was acceptable (Robinson, Shaver & Wrightsman, 1991; Schmitt, 1996; Sekaran, 2005). The low value of Cronbach’s alpha in the current study may be due to several influential factors, including the small number and heterogeneity of items on the MoCA, and little variance in scores from a non-clinical sample. This level of internal consistency is similar to what has been reported in a previous study of the MoCA in healthy, non-clinical populations (Berenstein et al., 2011), but lower than that reported in the original MoCA literature (Nasreddine et al., 2005). Completers versus non-completers did not differ on any variables except baseline RBANS. Completers demonstrated a significantly higher total RBANS scaled score compared to non-completers, as a result of significantly higher scores in the attention, immediate memory and delayed memory domains of the battery. These results indicate that change on RBANS over time may have been altered if these participants had been included.

Interestingly, only individuals scoring < 26 at baseline demonstrated a significant increase in MoCA scores from baseline to 12M. While individuals scoring ≥ 26 on the MoCA at baseline exhibited slightly decreased total MoCA scores from baseline to 12M, those scoring <26 exhibited increased scores by over 2.5 points on average. These results suggest that it was likely the change in performance of low-scoring individuals that led to the significant increase in scores between the first and second administrations, the test-retest interval where practice effects are typically observed.

Although 26 is the recommended cutoff to detect cognitive impairment on the MoCA (Nasreddine et al., 2005), subsequent studies have revealed questionable specificity of this criterion. For example, Luis et al. (2009) demonstrated that the MoCA cutoff score of 26 only had 35% specificity in identifying individuals with cognitive impairment. Alternatively, utilizing a cutoff score of 23 exhibited 95% specificity while retaining excellent sensitivity (96%). Results utilizing this cutoff were similar to the previous analysis. Individuals classified as low-scoring (≤ 23) significantly improved between the first and second administrations (baseline to 12M) but exhibited little change between the second and third administrations. High-scoring individuals did not demonstrate any significant change over time. However, it should be noted that group sizes were unequal and few scored below the cutoff. Other studies have suggested an even more conservative cutoff, such as ≤ 20 (Waldron-Perrine & Axelrod, 2012). However, no individuals in our sample scored below 22 on the MoCA, rendering an analysis utilizing this cutoff impossible.

Performance on the total RBANS score did not significantly change over time. This result corroborates previous research that identified no significant increases in performance on the RBANS over short retest intervals (Dong et al., 2013) and a one year retest interval (Duff et al., 2005). Additionally, increased performance was not expected on the RBANS in the current study due to the inclusion of alternate forms. When examining changes in performance between the first, second and third administrations, performance increased slightly between baseline and 12M, and slightly decreased from 12M to 24M testing. However, these changes were not significant.

Low baseline performance and subsequent improvements in cognitive performance are often attributed to practice effects. Previous studies have demonstrated that repeated assessment with a specific test results in an increase in performance at the time of the second assessment, and that performance stabilizes after the second testing period (Collie, Maruff, Darby, & McStephen, 2003; Falleti et al., 2006). Although the previous studies used short delays between testing periods (10 minutes – several hours), practice effects have been reported with intervals of over a year in participants who were cognitively normal, or diagnosed with MCI or dementia (Machulda et al., 2013). Similar practice effects have been demonstrated with significant increases over one-year retest intervals on tests of general cognition (Mathews, Abner, Caban-Holt, Kryscio & Schmitt, 2013) and on a memory screening measure (Abner et al., 2012).

Two alternate versions of the MoCA have recently been validated for use in repeated testing (Chertkow et al., 2011). These alternative forms demonstrated no practice effects over short (one month) intervals, but have yet to be examined in longer intervals. Similar results have been reported in the German language alternate MoCA versions over 60 minutes retest intervals (Costa et al., 2012). The alternate versions were not available at the beginning of the present longitudinal study and therefore the original form of the MoCA was given at all three testing points. Future research should examine the possible elimination of practice effects through the use of alternate MoCA forms over long retest intervals that are more representative of retest intervals in clinical situations. Additionally, there are situations where the option of alternate versions is not available, such as other language versions with no verified alternate forms or when the MoCA is administered to an individual at more than three time-points. Future research is needed to identify the retest interval that is sufficient for eliminating practice effects on the same version of the MoCA over time.

There are several important limitations of the study to consider. One consideration is the difference in retest intervals. A recent meta-analysis of practice effects suggests that shorter retest intervals are more prone to practice effects and the interval between the first and second administrations is likely to have the largest practice effects (Calamia et al., 2012). Although our results demonstrated the expected larger increase in scores between baseline and 12M testing, it is unclear if the non-significant change in scores from 12M to 48M is due to the larger retest interval. Another potential limitation is the number of individuals at baseline scoring below the cutoff of 26. As previously described, this cutoff may not exhibit optimal specificity in detecting cognitive impairment, resulting in individuals with normal cognitive functioning classified as low-scoring. Additionally, individuals with MCI or preclinical Alzheimer’s disease may have been enrolled in the present study. Although individuals in the current study were not given a diagnosis of MCI, the cutoff score of 26 has been previously utilized as an indicator of possible MCI. Previous research has suggested that individuals with amnestic MCI demonstrate larger practice effects on tests of delayed memory than cognitively normal peers over one week (Duff et al., 2008). Finally, the sample in the current study was relatively young and highly educated, limiting the generalizability of the results to the overall aging population.

The results of the present study indicate that the longitudinal pattern of performance on the MoCA does not exhibit an expected age-related decline. Performance on the MoCA increased from baseline to 12 month testing, and then remained stable until 48 months. This increase between the first and second administrations was primarily observed in individuals that scored low at baseline. These results suggest possible practice effects from first exposure to second exposure and therefore caution is warranted when interpreting performance on the MoCA longitudinally without the use of alternate forms.

Acknowledgements

This work was supported by the National Institutes of Health/National Institute of Neurological Disorders and Stroke under Grants [R01 NS052470, R01 NS039538]; National Institutes of Health /National Institute of Mental Health under Grant [R21 MH090494]; and National Institutes of Health /National Center for Research Resources under Grant [UL1 TR000448].

Footnotes

Disclosure Statement There are no financial interests to disclose.

References

  1. Abner EL, Dennis BC, Mathews MJ, Mendiondo MS, Caban-Holt A, Kryscio RJ, for the PREADViSE Investigators. Crowely JJ, for the SELECT Investigators Practice effects in a longitudinal, multi-center Alzheimer’s disease prevention clinical trial. Trials. 2012;13:217. doi: 10.1186/1745-6215-13-217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bernstein IH, Lacritz L, Barlow CE, Weiner MF, DeFina LF. Psychometric evaluation of the Montreal Cognitive Assessment (MoCA) in three diverse samples. The Clinical Neuropsychologist. 2011;25(1):119–126. doi: 10.1080/13854046.2010.533196. [DOI] [PubMed] [Google Scholar]
  3. Calamia M, Markon K, Tranel D. Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment. The Clinical Neuropsychologist. 2012;26(4):543–570. doi: 10.1080/13854046.2012.680913. [DOI] [PubMed] [Google Scholar]
  4. Chelune GJ, Naugle RI, Lüders H, Sedlak J, Awad IA. Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology. 1993;7(1):41–52. [Google Scholar]
  5. Chertkow H, Nasreddine Z, Johns E, Phillips N, McHenry C. The Montreal cognitive assessment (MoCA): Validation of alternate forms and new recommendations for education corrections [Supplemental material] Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association. 2011;7(4):S157. [Google Scholar]
  6. Collie A, Maruff P, Darby DG, McStephen M. The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test-retest intervals. Journal of the International Neuropsychological Society. 2003;9(3):419–428. doi: 10.1017/S1355617703930074. [DOI] [PubMed] [Google Scholar]
  7. Costa AS, Fimm B, Friesen P, Soundjock H, Rottschy C, Gross T, Reetz K. Alternate-form reliability of the Montreal Cognitive Assessment screening test in a clinical setting. Dementia and Geriatric Cognitive Disorders. 2012;33(6):379–384. doi: 10.1159/000340006. [DOI] [PubMed] [Google Scholar]
  8. Dong Y, Thompson CL, Tan S, Lim L, Pang W, Chen C. Test-retest reliability, convergent validity and practice effects of the RBANS in a memory clinic setting: a pilot study. Open Journal of Medical Psychology. 2013;2:11–16. [Google Scholar]
  9. Duff K, Beglinger LJ, Schoenberg MR, Patton DE, Mold J, Scott JG, Adams RL. Test-retest reliability and practice effects of the RBANS in a community dwelling elderly sample. Journal of Clinical and Experimental Neuropsychology. 2005;27(5):565–575. doi: 10.1080/13803390490918363. [DOI] [PubMed] [Google Scholar]
  10. Duff K, Humphreys-Clark JD, O’Bryant SE, Mold JW, Schiffer RB, Sutker PB. Utility of the RBANS in detecting cognitive impairment associated with Alzheimer’s disease: sensitivity, specificity, and positive and negative predictive powers. Archives of Clinical Neuropsychology. 2008;23(5):603–612. doi: 10.1016/j.acn.2008.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Duff K, Hobson VL, Beglinger LJ, O’Bryant SE. Diagnostic accuracy of the RBANS in mild cognitive impairment: limitations on assessing milder impairments. Archives of Clinical Neuropsychology. 2010;25(5):429–441. doi: 10.1093/arclin/acq045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Falleti MG, Maruff P, Collie A, Darby DG. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. Journal of Clinical and Experimental Neuropsychology. 2006;28(7):1095–1112. doi: 10.1080/13803390500205718. [DOI] [PubMed] [Google Scholar]
  13. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  14. Freitas S, Simões MR, Alves L, Santana I. Montreal cognitive assessment: Validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Disease and Associated Disorders. 2013;27(1):37–43. doi: 10.1097/WAD.0b013e3182420bfe. [DOI] [PubMed] [Google Scholar]
  15. Gagnon JF, Postuma RB, Joncas S, Desjardins C, Latreille V. The Montreal Cognitive Assessment: a screening tool for mild cognitive impairment in REM sleep behavior disorder. Movement Disorders: Official Journal of the Movement Disorder Society. 2010;25(7):936–940. doi: 10.1002/mds.23079. [DOI] [PubMed] [Google Scholar]
  16. Galasko D, Abramson I, Corey-Bloom J, Thal LJ. Repeated exposure to the Mini-Mental State Examination and the Information-Memory-Concentration Test results in a practice effect in Alzheimer’s disease. Neurology. 1993;43(8):1559–1563. doi: 10.1212/wnl.43.8.1559. [DOI] [PubMed] [Google Scholar]
  17. Helkala EL, Kivipelto M, Hallikainen M, Alhainen K, Heinonen H, Tuomilehto J, Nissinen A. Usefulness of repeated presentation of Mini-Mental State Examination as a diagnostic procedure – a population-based study. Acta Neurologica Scandinavica. 2002;106(6):341–346. doi: 10.1034/j.1600-0404.2002.01315.x. [DOI] [PubMed] [Google Scholar]
  18. Hoops S, Nazem S, Siderowf AD, Duda JE, Xie SX, Stern MB, Weintraub D. Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology. 2009;73(21):1738–1745. doi: 10.1212/WNL.0b013e3181c34b47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jacqmin-Gadda H, Fabrigoule C, Commenges D, Dartigues JF. A 5-year longitudinal study of the Mini-Mental State Examination in normal aging. American Journal of Epidemiology. 1997;145(6):498–506. doi: 10.1093/oxfordjournals.aje.a009137. [DOI] [PubMed] [Google Scholar]
  20. Luis CA, Keegan AP, Mullan M. Cross validation of the Montreal Cognitive Assessment in community dwelling older adults residing in the Southeastern US. International Journal of Geriatric Psychiatry. 2009;24(2):197–201. doi: 10.1002/gps.2101. [DOI] [PubMed] [Google Scholar]
  21. Machulda MM, Pankratz VS, Christianson TJ, Ivnik RJ, Mielke MM, Roberts RO, Petersen RC. Change in normal aging vs. incident mild cognitive impairment and dementia in the Mayo clinic study of aging. The Clinical Neuropsychologist. 2013;27(8):1247–1264. doi: 10.1080/13854046.2013.836567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Mathews M, Abner E, Caban-Holt A, Kryscio R, Schmitt F. CERAD practice effects and attrition bias in a dementia prevention trial. International Psychogeriatrics. 2014;25(7):1115–1123. doi: 10.1017/S1041610213000367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. McLennan SN, Mathias JL, Brennan LC, Stewart S. Validity of the Montreal cognitive assessment (MoCA) as a screening test for mild cognitive impairment (MCI) in a cardiovascular population. Journal of Geriatric Psychiatry and Neurology. 2011;24(1):83–88. doi: 10.1177/0891988710390813. [DOI] [PubMed] [Google Scholar]
  24. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, Chertkow H. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society. 2005;53:695–699. doi: 10.1111/j.1532-5415.2005.53221.x. [DOI] [PubMed] [Google Scholar]
  25. Olson RA, Chhanabhai T, McKenzie M. Feasibility study of the Montreal Cognitive Assessment (MoCA) in patients with brain metastases. Supportive Care in Cancer. 2008;16(11):1273–1278. doi: 10.1007/s00520-008-0431-3. [DOI] [PubMed] [Google Scholar]
  26. Pendlebury ST, Cuthbertson FC, Welch SJ, Mehta Z, Rothwell PM. Underestimation of cognitive impairment by Mini-Mental State Examination versus the Montreal Cognitive Assessment in patients with transient ischemic attack and stroke: a population-based study. Stroke. 2010;41(6):1290–1293. doi: 10.1161/STROKEAHA.110.579888. [DOI] [PubMed] [Google Scholar]
  27. Randolph C. Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) The Psychological Corporation; San Antonio: Harcourt, TX: 1998. [Google Scholar]
  28. Randolph C, Tierney MC, Mohr E, Chase TN. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology. 1998;20(3):310–319. doi: 10.1076/jcen.20.3.310.823. [DOI] [PubMed] [Google Scholar]
  29. Robinson JP, Shaver PR, Wrightsman LS. Criteria for scale selection and evaluation. In: Robinson JP, Wrightsman S, editors. Measures of personality and psychological attitudes. Academic Press; New York, NY: 1991. pp. 1–16. [Google Scholar]
  30. Schmitt N. Uses and abuses of coefficient alpha. Psychological Assessment. 1996;8(4):350–353. [Google Scholar]
  31. Sekaran U. Research Methods for Business: A Skill-building Approach. 4th ed John Wiley & Sons; New York: 2005. [Google Scholar]
  32. Toh EA, MacAskill MR, Dalrymple-Alford JC, Myall DJ, Livingston L, Macleod SA, Anderson TJ. Comparison of cognitive and UHDRS measures in monitoring disease progression in Huntington’s disease: A 12-month longitudinal study. Translational Neurodegeneration. 2014;3:15. doi: 10.1186/2047-9158-3-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Waldron-Perrine B, Axelrod BN. Determining an appropriate cutting score for indication of impairment on the Montreal Cognitive Assessment. International Journal of Geriatric Psychiatry. 2012;27:1189–1194. doi: 10.1002/gps.3768. [DOI] [PubMed] [Google Scholar]
  34. Wilkinson GS, Robertson GJ. Wide Range Achievement Test 4 Professional Manual. Psychological Assessment Resources; Lutz, FL: 2006. [Google Scholar]

RESOURCES