Abstract
Background
Cognitive complaints are one of the most frequent symptoms reported in post-acute sequelae of COVID-19 (PASC). The Montreal Cognitive Assessment (MoCA) has been used to estimate prevalence of cognitive impairment in many studies of PASC, and is commonly employed as a screening test in this population, however, its validity has not been established.
Objective
To determine the utility of the MoCA to screen for cognitive impairment in PASC.
Methods
Sixty participants underwent neuropsychological, psychiatric, and medical assessments, as well as the Montreal Cognitive Assessment, 6–8 months after acute COVID-19 infection.
Results
The overall sample had a mean score of 26.1 on the MoCA, with approximately one third screening below the cutoff score of 26, similar to the rate of extremely low NP test performance. MoCA score was inversely correlated with fatigue and depression measures and ethnic minority participants scored on average lower, despite similar education and estimated premorbid function. The MoCA had an accuracy of 63.3% at detecting any degree of diminished NP performance, and an accuracy of 73.3% at detecting extremely low NP performance.
Discussion/Conclusion
The MoCA may not be accurate for detecting neither mild nor more severe degrees of diminished NP test performance in PASC. Therefore, patients with persistent cognitive complaints in the setting of PASC who score in the normal range on the MoCA should be referred for formal NP assessment.
Keywords: COVID-19, Montreal cognitive assessment, Neuropsychological testing, Cognitive deficits, “Brain fog”
1. Introduction
According to the World Health Organization, SARS-CoV-2, the virus which causes COVID-19, has infected over 260 million people worldwide [1]. It is increasingly evident that even after recovering from the acute illness, many individuals experience unexpected long-term symptoms. These post-acute symptoms of COVID-19 (PASC) include fatigue, respiratory and cardiac manifestations, as well as cognitive and psychiatric symptoms.
Cognitive complaints reported by those suffering from PASC (often called “brain fog” by patients) include inattention, poor concentration, problems with memory and difficulties multitasking [2]. These symptoms have been found to frequently co-occur with psychiatric symptoms such as depression, anxiety, fatigue, and sleep disorders [2]. A comprehensive review of 12 studies of post-COVID-19 cognitive impairment found global impairment in 15–80% of participants, including deficits in memory, attention, executive function, and verbal fluency [3]. The studies included in this review relied primarily on screening measures such as the Montreal Cognitive Assessment (MoCA) and the Mini Mental State Exam (MMSE), with few using formal neuropsychological (NP) assessment [3]. Recent studies that employed formal NP testing had mixed findings, with some finding minimal to no deficits and others finding minimal deficits, attributing the deficits to other symptoms such as mood, poor sleep, and fatigue [4,5]. In contrast, a 2022 study from England examined post-hospitalized patients versus a control group found relatively severe cognitive impairment, which authors described as similar in magnitude to ageing 20 years [6].
This research group has reported neuropsychiatric findings from a cross-sectional assessment of 60 participants enrolled in a longitudinal investigation of neuropsychological, medical, and psychiatric sequelae of COVID-19. Results from the study indicated that over one-fourth (27%) of participants had extremely low NP test performance. The study also showed that individuals seeking care for cognitive complaints in the areas of memory, language and higher cognition, as measured in this by the Patient Assessment of Own Functioning (PAOF) inventory, scored significantly lower than population-based normative values on NP tests of attention, processing speed, memory, and executive function [7]. Independent predictors of impairment in this population included acute COVID-19 symptoms, current depression score, number of medical comorbidities, and subjective cognitive complaints in the areas of memory, language, and executive functions [7].
The above findings indicate that those who have recovered from COVID-19 infection may experience neurocognitive impairment, and that subjective cognitive complaints in the context of PASC warrant investigation. However, to the general post-COVID population seeking care, formal neuropsychological testing may be difficult to access, with limitations as far as cost, availability, and length of time to complete [8]. With this in mind, the investigators sought to examine if a commonly used screening tool for cognitive impairment, the Montreal Cognitive Assessment (MoCA), would be clinically useful at screening for neurocognitive deficits in patients with PASC, thus facilitating further investigation, including neuroimaging and formal testing [9].
The MoCA is a cognitive screening tool designed to be more sensitive for mild forms of cognitive impairment and specific cognitive domain functions compared to other measures such as the MMSE and has been validated in multiple populations [[10], [11], [12]]. Researchers worldwide have used the MoCA in the COVID-19 population to provide an estimate of cognitive functioning both during the acute and longer-term phases. One study examining the cognitive and psychological status of patients in the subacute and long-term phases of COVID-19 found that 70–75% of participants showed impairments on the MoCA [13]. A study from India utilized the MoCA to examine cognitive impairment after asymptomatic COVID-19 infection, finding impairment in both Total Score as well as in specific domains, namely visuospatial, naming, and verbal fluency [14]. A 2022 study from New York City screened 215 patients 6 months post-COVID-19 infection using a telephone version of the MoCA, finding that 49% had abnormal results, with predicting variables including older age, high school or lower level of education, unemployment prior to COVID, and Black race [15].
The MoCA does have limitations in this and other clinical populations. While the MoCA is widely utilized, it was designed as a screening tool that would trigger more formal neuropsychological (NP) testing, the gold standard. It may be less sensitive in detecting subtler forms of cognitive difficulty compared to NP testing. In population-based studies, this may lead to an under-estimate of actual cognitive impairment. In individual patients seen in clinical settings, mild impairment may be missed. In terms of the COVID-19 population, there are no studies investigating the utility of the MoCA as it compares contemporaneously to formal NP testing.
With this knowledge in mind, the investigators sought to assess individuals recovered from their acute COVID-19 illness and address the following:
-
1.
Whether the MoCA is an effective screening tool for neurocognitive impairment in individuals recovered from acute COVID-19?
-
2.
Whether psychiatric or medical measures correlate with MoCA performance?
2. Methods
This study was conducted at New York Medical College/Westchester Medical Center Health Network (WMCHealth), in Valhalla, NY. It was approved by the Institutional Review Board of New York Medical College (Protocol #14400) as well as the WMCHealth Clinical Research Institute. Data were obtained from the baseline assessment of 60 participants recruited for a longitudinal study of neurocognitive, medical, and psychiatric sequelae of COVID-19. Study participants were recruited via social media, flyers, email chains, and word-of-mouth. A subset of patients seeking care for “brain fog” were referred from the WMCHealth Post-COVID-19 Recovery Program. All interested persons were screened via telephone to determine eligibility for participation by investigators (SL, SS) based on the following criteria: 1) Age at least 20 years old; 2) documented positive COVID-19 nasopharyngeal test or positive antibody test prior to vaccination; 3) recovered from acute COVID-19 infection as per CDC recommendations (10–20 days after symptom onset and 24 h without fever); 4) completed minimum 8th grade education; 5) fluent in English; and 6) capable of signing informed consent. Persons with a prior diagnosis of a major neurocognitive disorder, traumatic brain injury with loss of consciousness, uncorrected visual/hearing deficits, intellectual disability, or unstable psychiatric symptoms were excluded.
Eligible participants met with the study assessors (SL, SS) who were trained to perform and score the assessment battery by co-PI (RD), a board-certified Neuropsychologist, and were supervised by the study PI (SF). During this visit, signed informed consent was obtained. Participants were compensated $40.00 for their time.
2.1. Study measurements and instruments
The primary measure of interest in this study was the MoCA, a 30-point neurocognitive screening test that takes approximately 10 min to complete. The test and administration instructions are available to clinicians online [16]. A score of 26 or above is generally considered to be “normal”, while any score below 26 indicates possible cognitive impairment. The MoCA assesses several cognitive domains, including short-term memory (two learning trials of five nouns and recall after 5 min); visuospatial abilities: clock-drawing and three-dimensional cube copy); executive function (a modified Trail Making B Test, phonemic fluency task, a two-item verbal abstraction task); language skills (three-item animal naming task, repetition of two syntactically complex sentences, and phonemic fluency). Attention, concentration, and working memory are assessed using a sustained attention task, serial subtraction, and a forward and backward digit span task. Also assessed is abstract reasoning, asking the participant to describe the similarity between two words. Orientation to time and place is assessed by asking the participant to state the date, city, and time of place in which the test is occurring [17].
In addition to the MoCA, the investigators obtained sociodemographic, medical, and psychiatric information from the participants. Sociodemographic measures included age, gender, ethnicity, relationship status, years of education and current employment. Medical measures included self-reported medical history and a detailed history of COVID-19 illness, including symptoms, treatment(s), hospitalization, and time since diagnosis. To measure COVID-19 symptom a Likert-type instrument was developed based on a published CDC Symptom questionnaire, and addressed 11 COVID-19 symptoms, each categorized as absent, mild, moderate, or severe, with score ranging from 0 to 33 [18]. Participants were asked to fill out the survey twice, once for their peak acute COVID-19 illness and once for the time of the appointment. Other validated medical, functional, psychiatric, and neurocognitive measures can be found in Table 1 . The neuropsychological battery consisted of measures assessing specific cognitive domains that have been implicated in other infectious and clinical disease states such as HIV, Lyme disease, cancer, and postural orthostatic tachycardia syndrome [[19], [20], [21], [22]].
Table 1.
Medical, Psychiatric, and Neurocognitive Testing Measures.
| Measure/Instrument | Description | Score Range |
|---|---|---|
| Medical/Functional Measures | ||
| Lawton-Brody Instrumental Activities of Daily Living Scale (IADL) | Measures practical aspect of everyday functioning [34] | 0–8 |
| Chalder Fatigue Scale (CFS-11) | An 11-item questionnaire which measures the severity of mental and physical fatigue [35] | 0–33. A cutoff score of >21 is considered clinically significant fatigue. |
| Psychiatric Measures | ||
| Patient Health Questionnaire-9 (PHQ-9) | Queries DSM-IV major depression criteria [36] | 0–27. Generally, a total score of 0–4 points indicates no/minimal depression, 5–9 indicates mild depression, 10–14 indicates moderate depression, 15–19 indicates moderately severe depression, and 20 or more indicating severe depression. For the purposes of this study, we used a cut off score of 10 or above to indicate clinically significant depression. |
| Post-Traumatic Stress Disorder Checklist for DSM-5 (PCL-5) | Queries DSM-V criteria for Post-Traumatic Stress Disorder (PTSD) [37] | 0–80, with a score of ≥33 indicating PTSD |
| Generalized Anxiety Disorder-7 (GAD-7) | Measures clinically significant anxiety [38] | 0–21, with a score ≥ 10 indicating a possible anxiety disorder |
| Neurocognitive Measures | ||
| Test of Premorbid Function (TOPF) | Estimates participants' intellectual functioning prior to an inciting event or injury [39] | 76–124 |
| Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Form A | A neuropsychological assessment designed to identify and characterize abnormal cognitive decline, which yields both a Total Score as well as sub-scores for five cognitive domains [40] | 40–160 |
| Trail Making Test Parts A and B | Measures psychomotor speed, visual search, and attention [41] | N/a |
| Verbal fluency (letter and category) | Assesses phonemic verbal fluency and semantic memory [42] | N/a |
| Stroop Color-Word Test | Assesses the ability to inhibit cognitive interference [43] | N/a |
| Patient Assessment of Own Functioning Inventory (PAOF) | Assesses participants' own perception of impairment in five domains. Subscales exist for: memory, language and communication, handedness, sensory-perception, and cognitive/intellectual functioning [44] | N/a |
Neuropsychological test scores were converted to t-scores according to their respective manuals and compared to age and education-adjusted (where available) population-based norms. We applied accepted clinical practice for assessing NP test performance: “Low” NP test performance was defined as receiving a score one or more standard deviations below age-matched population-based normative values on two or more NP tests. “Extremely Low” NP test performance was defined as scores ≥2 standard deviations below (≤2nd percentile) below normative values (age and education-adjusted) on one or more of the 11 tests [23].
Data were analyzed using SPSS software [24]. These included descriptive statistics (frequency, mean, standard deviation); Chi-square for group comparisons on categorical variables; independent and one-sample t-tests and analysis of covariance (ANCOVA) for group comparisons on continuous variables. Pearson correlations were used to explore associations between various test scores and clinical variables. Finally, logistic regression was used to identify independent predictors of NP impairment.
3. Results
3.1. Overall sample
Sociodemographic information regarding this sample can be found in Table 2 . On average, participants were approximately 40 years of age. Slightly over half of the patients identified as White, about two thirds were female, two thirds in a relationship, and over three fourths were employed. Participants were on average approximately 7 months out from their acute illness. Approximately 40% had a prior psychiatric history (Table 2).
Table 2.
Comparison of Participants Scoring Above and Below Threshold on the MoCA.
| Variable | Total Sample (N = 60) |
MoCA ≥ 26 (N = 38) |
MoCA < 26 (N = 22) |
P-Value |
|---|---|---|---|---|
| Sociodemographic Factors | ||||
| Age (Mean, SD) | 41.4 (13.9) | 37.2 (12.1) | 48.5 (14.2) | p < 0.01 |
| Race (N, %) | p = 0.03 | |||
| White | 35 (58.3%) | 27 (71.1%) | 8 (36.4%) | |
| Black | 5 (8.3%) | 1 (2.6%) | 4 (18.2%) | |
| Hispanic | 13 (21.7%) | 6 (15.8%) | 7 (31.8%) | |
| Asian/South Asian | 5 (8.3%) | 2 (5.3%) | 3 (13.6%) | |
| Other | 2 (3.3%) | 2 (5.3%) | 0 (0.0%) | |
| Employed (N, %) | 50 (83.3%) | 34 (89.5%) | 16 (72.7%) | p = 0.09 |
| In Relationship or Married (N, %) | 38 (63.3%) | 23 (60.5%) | 15 (68.2%) | p = 0.55 |
| Medical Factors | ||||
| Peak COVID Symptom Score (Mean, SD) | 16.5 (6.1) | 15.4 (6.1) | 18.3 (5.9) | p = 0.08 |
| Appointment 1 COVID Symptom Score (Mean, SD) | 5.7 (4.6) | 4.8 (4.3) | 7.2 (4.7) | p = 0.06 |
| Psychiatric Factors | ||||
| Prior Psychiatric History | 24 (40.0%) | 5 (22.7%) | 19 (50.0%) | p = 0.04 |
| PHQ-9 > 11 (N, %) | 30 (50.0%) | 15 (39.5%) | 15 (68.2%) | p = 0.03 |
| GAD-7 > 10 (N, %) | 17 (28.3%) | 11 (28.9%) | 6 (27.3%) | p = 0.89 |
| PCL-5 > 33 (N, %) | 12 (20.0%) | 6 (15.8%) | 6 (27.3%) | p = 0.28 |
| Chalder >21 (N, %) | 30 (50.0%) | 15 (39.5%) | 15 (68.2%) | p = 0.09 |
| Clinical Grouping | ||||
| None-Mild NP Test Impairment (N, %) | 44 (73.3%) | 33 (86.8%) | 11 (50.0%) | p < 0.01 |
| Severe NP Test Impairment (N, %) | 16 (26.7%) | 5 (13.2%) | 11 (50.0%) | |
| Non-Clinical Group (N, %) | 28 (46.7%) | 23 (82.1%) | 5 (17.9%) | p < 0.01 |
| Clinical Group (N, %) | 32 (53.3%) | 15 (46.9%) | 17 (53.1%) | |
| MoCA Subscores | ||||
| MoCA Visuospatial (Max = 5) (Mean, SD) | 4.5 (0.7) | 4.7 (0.5) | 4.1 (0.8) | p < 0.01 |
| MoCA Naming (Max = 3) (Mean, SD) | 2.9 (0.3) | 2.9 (0.3) | 2.8 (0.4) | p = 0.41 |
| MoCA Attention (Max = 6) (Mean, SD) | 5.7 (1.0) | 6.0 (0.2) | 5.2 (1.4) | p = 0.02 |
| MoCA Language (Max = 3) (Mean, SD) | 2.6 (0.7) | 2.9 (0.4) | 2.1 (0.9) | p < 0.01 |
| MoCA Abstraction (Max = 2) (Mean, SD) | 1.8 (0.4) | 1.8 (0.4) | 1.8 (0.4) | p = 0.63 |
| MoCA Delayed Recall (Max = 5) (Mean, SD) | 2.6 (1.6) | 3.5 (1.2) | 1.1 (1.0) | p < 0.01 |
| MoCA Orientation (Max = 6) (Mean, SD) | 6.0 (0.2) | 6.0 (0.2) | 6.0 (0.0) | p = 0.16 |
3.2. MoCA
The group of 60 had a mean total MoCA of 26.1, with a standard deviation of 2.6. In this sample, 22 participants (36.7%) scored below a 26 on the MoCA (screened positive), indicating possible neurocognitive impairment (Table 2 ). The subcategory in which participants scored lowest was Delayed Recall, with only 10 participants scoring a 5/5 on the task. Participants who scored <26 on the Total MoCA scored lower on the subcategories of Visuospatial, Attention, Language, and Delayed Recall than those scoring above the cutoff.
3.3. MoCA and sociodemographic factors
When comparing participants who scored <26 MoCA vs ≥ 26, we found a significant relationship with race (Table 2). While only 22.9% of the 35 White participants scored below a 26 on the MoCA, 56.0% of the 25 non-White participants (Black, Hispanic, Asian/South Asian) screened as impaired.
3.4. MoCA and medical factors
There was a significant correlation between Total MoCA Score and Chalder Fatigue Total Score (r = −0.264, p < 0.01), Chalder Fatigue Physical Fatigue Score (r = −0.397, p < 0.01), Chalder Fatigue Mental Fatigue Score (r = −0.264, p = 0.04) and IADL score (r = 0.367, p < 0.01). No significant relationship was found between impairment on the MoCA and acute or current appointment COVID symptom score, time from acute COVID illness, date of testing, or number of medical comorbidities. (Table 2).
3.5. MoCA and psychiatric measures
Fewer participants with self-reported prior psychiatric history scored <26 on the MoCA compared to those with no reported psychiatric history (p = 0.04). However, participants who had clinically significant depression based on the PHQ-9 were statistically more likely to score < 26 on the MoCA (p = 0.03). No significant differences in substance-use history, clinically significant anxiety (based on the GAD-7), clinically significant PTSD (based PCL-5), or psychotropic medication use was found (Table 2).
3.6. MoCA and NP testing
Within the Normal NP Test group, 16.7% (4 participants) scored <26 on the MoCA, compared to 35% (7 participants) of the Low NP Test group, and 68.8% (11 participants) of the Extremely Low NP Test group. There was a statistically significant relationship (p < 0.01) between scoring <26 on the MoCA and NP test group (Normal, Low, Extremely Low). However, in post-hoc analysis, there was no statistical difference between those with Normal NP test performance and those with Low NP test performance (p = 0.16), while statistically significantly fewer participants with Normal NP Test performance had a MoCA <26 than those with Extremely Low NP Test performance (p < 0.01). When we combined the groups with Normal NP test performance and those with Low NP Test performance as compared to those with Extremely Low NP Test performance, the significant difference remained (p < 0.01).
MoCA Total and Subcategory mean scores between those with Normal/Low NP test performance were then compared with those with Extremely Low NP test performance, finding significant differences on Total MoCA (26.8 vs 24.1; p < 0.01), Naming (2.9 vs 2.7; p = 0.01), Language (2.8 vs 2.3; p = 0.01), and Delayed Recall (2.9 vs 1.8; p = 0.02).
Next, the investigators sought to define the MoCA's ability to differentiate Normal NP test scores from any degree of poor NP test performance (Low and Extremely Low), as well as combined Normal/Low NP test performance from Extremely Low NP test performance. We found the MoCA's accuracy at detecting any degree of poor NP test performance to be 63.3%, with a sensitivity of 50.0%, specificity of 83.3%, positive predictive value of 81.8%, and a negative predictive value of 52.6%. However, when looking at the ability of the MoCA to differentiate between Normal/Low NP Test performance vs. Extremely Low NP Test performance, we found the MoCA to have an accuracy of 73.3%, sensitivity of 68.8%, specificity of 75.0%, positive predictive value of 50.0%, and a negative predictive value of 86.8%.
When looking at the MoCA's accuracy at detecting Extremely Low as opposed to Normal/Low, we found that it was significantly more accurate in White versus non-White participants. Accuracy among Black (60.0%), Hispanic (53.8%), and Asian/South Asian (60.0%) participants were similar. When the three minority categories were combined, the accuracy of the MoCA in the non-White group was 60.0%, significantly lower than the White participant group (82.9%) (p = 0.048). These two groups also differed significantly in terms of mean TOPF score, with White participants scoring significantly higher than non-White participants (111.9 vs 104.4, respectively, p = 0.01), though both group means fell in the High-Average to Average range. There were no differences as far as years of education.
In order to graphically depict both the linearity of the relationship and which MoCA cut-off score might be most sensitive to detect each degree of NP test performance, we constructed a generalized additive model, allowing us to characterize the relationship between total MoCA score and level of NP test performance (Normal, Low, and Extremely Low) (Fig. 1 ). This graph illustrates a linear relationship of MoCA score with level of NP test score below the clinical cutoff score of 26, with lower score on MoCA corresponding to progressively higher likelihood of Extremely Low NP test score. Above the score of 26, the curve flattens, and no such relationship exists.
Fig. 1.
Non-linear regression model.
Generalized additive model to show non-linear univariate regression modeling of MoCA scores to predict poor NP test performance (EDF = 4.3, R2 = 0.26, p = 0.15).
To determine the adjusted odds ratio for degree of NP test performance (Normal/Low vs Extremely Low) based on incremental decline in MOCA total score, we conducted a logistic regression, adjusting for age, severity of acute COVID-19 symptoms, number of medical comorbidities, prior psychiatric history, education, and gender. Using backward stepwise elimination, we found that the only significant predictors of Extremely Low NP test performance versus Normal/Low NP test performance were total MoCA score and gender (p < 0.01, OR = 0.53; and p = 0.05, OR = 0.21, respectively) (Table 3 ). This model found that for every unit increase in Total MoCA Score, the odds of Extremely Low NP test performance decrease by 47%. Further, being of female gender decreases the odds of scoring Extremely Low by 79%.
Table 3.
Multivariate Logistic Regression with Backward Elimination Predicting Odds of Extremely Low NP Test Performance versus Normal/Low NP Test Performance.
| Variable | Odds Ratio | 95% Confidence Interval (Lower Bound) | 95% Confidence Interval (Upper Bound) | P-value |
|---|---|---|---|---|
| Gender | 0.21 | 0.04 | 0.98 | 0.048 |
| Prior Psychiatric History | 0.23 | 0.05 | 1.13 | 0.07 |
| Total MoCA Score | 0.53 | 0.38 | 0.76 | < 0.01 |
| Age | Removed by backwards stepwise elimination. | |||
| Years of Education | ||||
| Peak COVID Symptom Score | ||||
| Number of Co-Morbid Medical Conditions | ||||
In terms of the participants' subjective cognitive complaints, we found that Total MoCA score significantly correlated with the Patient Assessment of Own Functioning Inventory (PAOF), specifically within the domains of Memory (r = −0.49, p < 0.001) and Cognitive/Intellectual Functioning (r = −0.41, p < 0.001). The correlation between PAOF Cognitive/Intellectual Functioning and Total MoCA was not present in the Normal/Low NP Test performance group (r = −0.22, p = 0.15), and though the correlation with Memory remained, it was less marked (r = −0.34, p = 0.03).
4. Discussion
Data from this sample suggests that based on the MoCA alone, a significant portion of study participants may suffer from neurocognitive impairment after recovering from acute COVID-19 infection, as over one third of all participants scored below a 26. This finding is unsurprising given the numerous prior reports demonstrating post-COVID cognitive impairment [[2], [3], [4]]. When looking at the subcategories of the MoCA, we found that participants scored lowest in Delayed Recall, which has also been found in more comprehensive neurocognitive testing reports, as well as other studies looking at the MoCA's use post-COVID [7,25]. Unlike in our prior NP testing findings, the MoCA did not identify significant impairment in attention, processing speed, abstraction, or executive function [7]. This may be in part because the MoCA does not effectively measure processing speed or executive function, as it uses a modified version of the Trail Making Part B which does not involve timing the participant, and only one letter for the phonemic fluency portion of the test as opposed to the normal three [16]. However, on a subjective measure of impairment post-COVID, we previously found that participants reported memory as one of their main complaints, so it is promising that the MoCA results reflected this [7].
Results regarding accuracy of the MoCA in detecting poor NP test performance were mixed. We found that when attempting to detect Low/Extremely Low test performance vs. Normal NP test performance, the MoCA was only 63.3% accurate and 50% sensitive. When we combined Normal performance and Low NP Test performance and differentiated from Extremely Low NP test performance, the MoCA was slightly more accurate, increasing the accuracy to 73.3% and sensitivity to 68.8%. This suggests that the MoCA may not be able to detect mild degrees of impairment in patients post-COVID-19 infection, and may only be marginally better at detecting more severe degrees of impairment. This is noteworthy because the MoCA is described as being more accurate and sensitive than other similar screening measures, such as the MMSE, at detecting mild impairment, both in general and in post-COVID patients. Some have even suggested a telephone version of the MoCA could detect impairment accurately [26]. However, the majority of these prior studies did not directly compare MoCA results with a comprehensive NP testing post-COVID-19 infection. Neuropsychologists have historically examined what the most sensitive cutoff scores on the MoCA should be for various medical/neuropsychiatric conditions, with the consensus of <26 being most accurate. Our general additive model appeared to verify this in the current sample, so we did not attempt to deviate from this norm. Our findings suggest that given its low sensitivity for Low NP test performance, if patients present with post-COVID cognitive complaints and score as “normal” on the MoCA, a formal neuropsychological assessment referral should be considered.
When looking at sociodemographic factors that correlated with total MoCA score, we found a significant relationship with race. On average, non-White participants had lower overall MOCA scores and were more likely to score as impaired (below 26). This contrasts with our prior findings, which showed no racial/ethnic differences in rates of poor NP test performance using a NP testing battery, though is consistent with other reports finding that Black race may be a predictor of poor performance on the MoCA [7,15].Further, if looking at level of accuracy of the MoCA predicting poor NP test performance for White compared to Black, Hispanic or both combined, the MoCA was significantly more accurate at predicting poor performance for White participants, accurately detecting low NP test scores in 82.9% of White participants but only 60% in non-White participants.
There is literature suggesting that cutoff scores for detecting NP impairment on the MoCA may differ based on race/ethnicity. Some researchers have suggested that optimal cutoffs for detection of mild cognitive impairment would be 25 for non-Hispanic Whites, 24 among Hispanics and 23 among non-Hispanic Blacks [27]. While our sample size was small for non-Whites, we looked at accuracy utilizing the above cutoffs and found that it did not change significantly for either detecting Low/Extremely Low NP test performance vs. Normal performance (decrease in about 2%), or for Normal/Low NP test performance vs. Extremely Low NP test performance (increase in about 5%). Additionally, our generalized additive model in Fig. 1 suggests that the current cut-off score of 26 remains the most predictive. It's difficult to postulate on potential explanations for this phenomenon given that only 25 participants in this study were Non-White. Prior studies have discussed the limitations of using the MoCA cross-culturally, although most have examined the accuracy of alternative, non-English translations of the MoCA as opposed to comparing how those with different racial backgrounds score on one particular form [28].
We found that higher levels of physical and mental fatigue as measured by the Chalder Fatigue scale was correlated with lower MoCA score. This is in accordance with reports of mental fatigue correlating strongly with cognitive slowness detected by the MoCA (26). Interestingly, we found no relationship between MoCA and scores on the COVID-19 symptom scale, either during acute illness or at time of administration. This contrasts with other published reports which indicate that more severe COVID symptoms at time of infection were associated with poorer NP performance as well as lower scores on the MoCA [29,30].
Psychiatrically, we found that participants with clinically significant depression performed poorer on the MoCA. This is consistent with the abundance of literature suggesting depression is associated with cognitive impairment, even as indicated by MoCA score < 26 [31]. Additionally, the authors have previously reported that depressive symptoms were independently predictive of Extremely Low NP test scores [7]. It is not possible to distinguish if patients are experiencing depressive symptoms due to NP impairment or vice versa. This causal relationship of depression and cognitive impairment post-COVID remains an important area of research inquiry. Counter-intuitively, participants in this cohort with prior psychiatric history were less likely to score as impaired on the MoCA. This was unexpected, as we know that patients with psychiatric issues were more affected by COVID-19 than other patients, and even have been shown to have a higher mortality rate [32].
Our study is unique in that we compared the MoCA with concurrently administered NP test findings in participants that have recovered from acute COVID-19 illness. To our knowledge, no other similar reports have been published. However, our study was not without limitations. Our sample was relatively small, limiting the power of our statistical analyses. This study included a range of patients, including patients from the community with no post-covid complaints, as well people seeking care from a specialized Post-COVID recovery program, allowing us to examine the use of the MoCA across a wide range of symptom burden. On the other hand, this may also represent a strength of this study because the MoCA is most likely to be applied in a clinical setting.
5. Conclusion
The MoCA may not be an adequate measure for screening for cognitive complaints in PASC. Patients with cognitive complaints should be referred for appropriate NP testing and treatment. The NP battery used in this investigation assesses multiple cognitive domains in a more thorough degree than the MoCA and takes under one hour to complete. Prior studies have shown that MoCA performance post-COVID-19 infection improves over time [33]. Longitudinal follow-up of this cohort is in progress, with plans to re-administer the MoCA and NP testing at 6 months and 18 months after the initial appointment.
CRediT authorship contribution statement
Sean Lynch: Conceptualization, Methodology, Formal analysis, Writing – original draft, Project administration. Stephen J. Ferrando: Conceptualization, Methodology, Formal analysis, Writing – original draft, Project administration, Supervision. Rhea Dornbush: Conceptualization, Methodology, Writing – review & editing, Project administration, Supervision. Sivan Shahar: Resources, Data curation, Writing – review & editing, Project administration. Abbas Smiley: Formal analysis, Data curation, Writing – review & editing. Lidia Klepacz: Resources, Supervision, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Funding for the current study is from the Edith Har Esh, M.D. Professorship Endowment Fund – New York Medical College. Funds were utilized to cover costs of participant reimbursement, laboratory assays and neuropsychological test materials.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.genhosppsych.2022.07.013.
Appendix A. Supplementary data
Supplemental Figure 1. Bar graph depicting distribution frequencies of total MoCA scores; Supplemental Figure 2. Bar graph depicting percent of participants with impaired scores in MoCA domains, comparing those with MoCA scores greater than or equal to 26 and those that scored below 26; Supplemental Table 1. Comparison between participants with Normal/Low NP test performance and those with Extremely Low NP Test performance across MoCA subscores; Supplemental Table 2. Description of MoCA accuracy across various groups (clinical vs non-clinical and White vs non-White).
References
- 1.World Health Organization. (n.d.). Coronavirus disease (covid-19). World Health Organization. Retrieved May 30, 2022, from https://www.who.int/emergencies/diseases/novel-coronavirus-2019.
- 2.Ferrarese C., Silani V., Priori A., Galimberti S., Agostoni E., Monaco S., et al. An Italian multicenter retrospective-prospective observational study on neurological manifestations of covid-19 (NEUROCOVID) Neurol Sci. 2020;41(6):1355–1359. doi: 10.1007/s10072-020-04450-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Daroische R., Hemminghyth M.S., Eilertsen T.H., Breitve M.H., Chwiszczuk L.J. Cognitive impairment after covid-19—a review on objective test data. Front Neurol. 2021;12 doi: 10.3389/fneur.2021.699582. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Krishnan K., Miller A.K., Reiter K., Bonner-Jackson A. Neurocognitive profiles in patients with persisting cognitive symptoms associated with covid-19. Arch Clin Neuropsychol. 2022;37(4):729–737. doi: 10.1093/arclin/acac004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Whiteside D.M., Basso M.R., Naini S.M., Porter J., Holker E., Waldron E.J., et al. Outcomes in post-acute sequelae of COVID-19 (PASC) at 6 months post-infection part 1: cognitive functioning. Clin Neuropsychol. 2022;36(4):806–828. doi: 10.1080/13854046.2022.2030412. [DOI] [PubMed] [Google Scholar]
- 6.Hampshire A., Chatfield D.A., MPhil A.M., Jolly A., Trender W., Hellyer P.J., et al. Multivariate profile and acute-phase correlates of cognitive deficits in a COVID-19 hospitalised cohort. EClinicalMedicine. 2022;47 doi: 10.1016/j.eclinm.2022.101417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ferrando S.J., Dornbush R., Lynch S., Shahar S., Klepacz L., Karmen C.L., et al. Neuropsychological, medical, and psychiatric findings after recovery from acute COVID-19: a cross-sectional study. J Acad Consul Liaison Psych. 2022 doi: 10.1016/j.jaclp.2022.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Neurodynamics. (n.d.). Retrieved May 30, 2022, from https://neurodynamics.biz/home/.
- 9.Nasreddine Z.S., Phillips N.A., Bédirian V.Ã., Charbonneau S., Whitehead V., Collin I., et al. The Montreal cognitive assessment, MOCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695–699. doi: 10.1111/j.1532-5415.2005.53221.x. [DOI] [PubMed] [Google Scholar]
- 10.Larner A.J. Screening utility of the Montreal cognitive assessment (MOCA): in place of – or as well as – the MMSE? Int Psychogeriatr. 2011;24(3):391–396. doi: 10.1017/s1041610211001839. [DOI] [PubMed] [Google Scholar]
- 11.Aggarwal A., Kean E. Comparison of the folstein mini mental state examination (MMSE) to the montreal cognitive assessment (MOCA) as a cognitive screening tool in an inpatient rehabilitation setting. Neurosci Med. 2010;01(02):39–42. doi: 10.4236/nm.2010.12006. [DOI] [Google Scholar]
- 12.Aggarwal A., Kean E. Comparison of the folstein mini mental state examination (MMSE) to the montreal cognitive assessment (MOCA) as a cognitive screening tool in an inpatient rehabilitation setting. Neurosci Med. 2010;01(02):39–42. doi: 10.4236/nm.2010.12006. [DOI] [Google Scholar]
- 13.Pistarini C., Fiabane E., Houdayer E., Vassallo C., Manera M.R., Alemanno F. Cognitive and emotional disturbances due to covid-19: an exploratory study in the rehabilitation setting. Front Neurol. 2021;12 doi: 10.3389/fneur.2021.643646. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Amalakanti S., Arepalli K.V., Jillella J.P. Cognitive assessment in asymptomatic covid-19 subjects. VirusDisease. 2021;32(1):146–149. doi: 10.1007/s13337-021-00663-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Valdes E., Fuchs B., Morrison C., Charvet L., Lewis A., Thawani S., et al. Demographic and social determinants of cognitive dysfunction following hospitalization for covid-19. J Neurol Sci. 2022;120146 doi: 10.1016/j.jns.2022.120146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Maust D., Cristancho M., Gray L., Rushing S., Tjoa C., Thase M.E. Psychiatric rating scales. Neurobiol Psych Dis. 2012;227–37 doi: 10.1016/b978-0-444-52002-9.00013-9. [DOI] [PubMed] [Google Scholar]
- 17.Rossetti H.C., Lacritz L.H., Cullum C.M., Weiner M.F. Normative data for the Montreal cognitive assessment (MOCA) in a population-based sample. Neurology. 2011;77(13):1272–1275. doi: 10.1212/wnl.0b013e318230208a. [DOI] [PubMed] [Google Scholar]
- 18.Centers for Disease Control and Prevention. (n.d.). Symptoms of COVID-19. Centers for Disease Control and Prevention. Retrieved May 30, 2022, from http://www.cdc.gov/coronavirus/2019-ncov/symptoms-testing/symptoms.html.
- 19.Ferrando S.J. Diagnosis and treatment of HIV-associated neurocognitive disorders. New Dir Ment Health Serv. 2000;2000(87):25–35. doi: 10.1002/yd.23320008705. [DOI] [PubMed] [Google Scholar]
- 20.Stefano G.B. Historical insight into infections and disorders associated with neurological and psychiatric sequelae similar to long Covid. Med Sci Monit. 2021;27 doi: 10.12659/msm.931447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bransfield R.C., Aidlen D.M., Cook M.J., Javia S. A clinical diagnostic system for late-stage neuropsychiatric Lyme Borreliosis based upon an analysis of 100 patients. Healthcare. 2020;8(1):13. doi: 10.3390/healthcare8010013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jansen C.E., Miaskowski C.A., Dodd M.J., Dowling G.A. A meta-analysis of the sensitivity of various neuropsychological tests used to detect chemotherapy-induced cognitive impairment in patients with breast cancer. Oncol Nurs Forum. 2007;34(5):997–1005. doi: 10.1188/07.onf.997-1005. [DOI] [PubMed] [Google Scholar]
- 23.Van Gorp W. Methodologic issues in neuropsychological research with HIV-spectrum disease. Arch Clin Neuropsychol. 1993;8(1):17–33. doi: 10.1016/0887-6177(93)90040-8. [DOI] [PubMed] [Google Scholar]
- 24.Corp, I. B. M . IBM Corp; Armonk, NY: 2013. IBM SPSS statistics for windows, version 22.0. [Google Scholar]
- 25.Beaud V., Crottaz-Herbette S., Dunet V., Vaucher J., Bernard-Valnet R., Du Pasquier R., et al. Pattern of cognitive deficits in severe COVID-19. J Neurol Neurosurg Psychiatry. 2020;92(5):567–568. doi: 10.1136/jnnp-2020-325173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Larner A.J. Cognitive testing in the COVID-19 ERA: can existing screeners be adapted for telephone use? Neurodegenerat Dis Manag. 2021;11(1):77–82. doi: 10.2217/nmt-2020-0040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Milani S.A., Marsiske M., Cottler L.B., Chen X., Striley C.W. Optimal cutoffs for the Montreal cognitive assessment vary by race and ethnicity. Alzheimer’s Dementia: Diag Assess Dis Monitor. 2018;10(1):773–781. doi: 10.1016/j.dadm.2018.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.O’Driscoll C., Shaikh M. Cross-cultural applicability of the montreal cognitive assessment (MOCA): a systematic review. J Alzheimers Dis. 2017;58(3):789–801. doi: 10.3233/jad-161042. [DOI] [PubMed] [Google Scholar]
- 29.Cristillo V., Pilotto A., Cotti Piccinelli S., Bonzi G., Canale A., Gipponi S., et al. Premorbid vulnerability and disease severity impact on long-covid cognitive impairment. Aging Clin Exp Res. 2022;34(1):257–260. doi: 10.1007/s40520-021-02042-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Miskowiak K.W., Johnsen S., Sattler S.M., Nielsen S., Kunalan K., Rungby J., et al. Cognitive impairments four months after covid-19 hospital discharge: pattern, severity and association with illness variables. Eur Neuropsychopharmacol. 2021;46:39–48. doi: 10.1016/j.euroneuro.2021.03.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Blair M., Coleman K., Jesso S., Desbeaumes Jodoin V., Smolewska K., Warriner E., et al. Depressive symptoms negatively impact montreal cognitive assessment performance: a memory clinic experience. Can J Neurol Sci J Can Des Sci Neurol. 2016;43(4):513–517. doi: 10.1017/cjn.2015.399. [DOI] [PubMed] [Google Scholar]
- 32.Nemani K., Li C., Olfson M., Blessing E.M., Razavian N., Chen J., et al. Association of psychiatric disorders with mortality among patients with COVID-19. JAMA Psychiat. 2021;78(4):380. doi: 10.1001/jamapsychiatry.2020.4442. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Del Brutto O.H., Rumbea D.A., Recalde B.Y., Mera R.M. Cognitive sequelae of long covid may not be permanent: a prospective study. Eur J Neurol. 2021;29(4):1218–1221. doi: 10.1111/ene.15215. [DOI] [PubMed] [Google Scholar]
- 34.Graf C. The Lawton instrumental activities of daily living scale. AJN, Am J Nurs. 2008;108(4):52–62. doi: 10.1097/01.naj.0000314810.46029.74. [DOI] [PubMed] [Google Scholar]
- 35.Jackson C. The Chalder fatigue scale (CFQ 11) Occup Med. 2015;65(1):86. doi: 10.1093/occmed/kqu168. [DOI] [PubMed] [Google Scholar]
- 36.Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9. J Gen Intern Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Blevins C.A., Weathers F.W., Davis M.T., Witte T.K., Domino J.L. The posttraumatic stress disorder checklist for DSM-V (PCL-5): development and initial psychometric evaluation. J Trauma Stress. 2015;28(6):489–498. doi: 10.1002/jts.22059. [DOI] [PubMed] [Google Scholar]
- 38.Williams N. The GAD-7 questionnaire. Occup Med. 2014;64(3):224. doi: 10.1093/occmed/kqt161. [DOI] [Google Scholar]
- 39.Wechsler D. 2008. Wechsler Adult Intelligence Scale--Fourth Edition. PsycTESTS Dataset. [DOI] [Google Scholar]
- 40.Randolph C., Tierney M.C., Mohr E., Chase T.N. The repeatable battery for the assessment of neuropsychological status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol. 1998;20(3):310–319. doi: 10.1076/jcen.20.3.310.823. [DOI] [PubMed] [Google Scholar]
- 41.Lezak M.D., Howieson D.B., Loring D.W., Hannay H.J., Fischer J.S. Oxford University press; 2004. Neuropsychological Assessment. [Google Scholar]
- 42.Gladsjo J.A., Schuman C.C., Evans J.D., Peavy G.M., Miller S.W., Heaton R.K. Norms for letter and category fluency: demographic corrections for age, education, and ethnicity. Assessment. 1999;6(2):147–178. doi: 10.1177/107319119900600204. [DOI] [PubMed] [Google Scholar]
- 43.Golden C.J., Freshwater S.M. Stroop color and word test: a manual for clinical and experimental uses. Stoelting. 2002 [Google Scholar]
- 44.Chelune C.J., Heaton R.K., Lehman R.A.W. Neuropsychological and personality correlates of patients’ complaints of disability. Adv Clin Neuropsychol. 1986;3:95–126. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Figure 1. Bar graph depicting distribution frequencies of total MoCA scores; Supplemental Figure 2. Bar graph depicting percent of participants with impaired scores in MoCA domains, comparing those with MoCA scores greater than or equal to 26 and those that scored below 26; Supplemental Table 1. Comparison between participants with Normal/Low NP test performance and those with Extremely Low NP Test performance across MoCA subscores; Supplemental Table 2. Description of MoCA accuracy across various groups (clinical vs non-clinical and White vs non-White).