Comparison of Oxford Cognitive Screen and Montreal Cognitive Assessment feasibility in the stroke unit setting. A pilot study

Highlights

• •MoCA and OCS are brief tools to assess cognition in stroke patients.
• •We found good agreement between MoCA and OCS measures.
• •MoCA and OCS were feasible in the stroke unit setting.
• •Qualitative information related to the patients and setting were registered.
• •We found high level of acceptability by patients for both tests.

Abstract

Background

: Cognitive status evaluation is not routine in the acute stroke setting and there is no consensus on which neuropsychological tool is more feasible and informative. The aim of this pilot study was to compare the feasibility and acceptability of two brief cognitive tests, the Montreal Cognitive Assessment (MoCA) and the Oxford Cognitive Screen (OCS), in acute stroke, with a focus on patients' experience, administration time, and the cognitive data obtained.

Methods

: Patients with a diagnosis of ischemic or hemorrhagic stroke or of transient ischemic attack admitted to two stroke units were included. The sample consisted of 34 participants (mean age ±SD 71.1 ± 16.1 years, 25 males). Within five days of onset, patients were evaluated by means of the MoCA and OCS by a trained neuropsychologist.

Results

Both tests were feasible in the stroke unit setting and had a high level of acceptability by patients. MoCA test was fully completed by 25 patients, OCS by 21 patients. The OCS administration time was longer than that of the MoCA. However, OCS was perceived less demanding than MoCA by patients. Twenty patients completed both the MoCA and the OCS entirely, and only 2 of them did not show any impairment in both tests. Seventeen patients showed at least an impaired domain on the OCS and 15 patients presented with a MoCA global score below cut-off for cognitive impairment.

Conclusions

Our preliminary study did not show a superiority of the OCS over the widely used MoCA, and suggests the need for further validation in larger samples of stroke patients, exploring tests accuracy in detecting cognitive post-stroke impairment.

1. Introduction

Post-stroke cognitive impairment (PSCI) frequently occurs in patients with stroke, and encompasses a large variety of cognitive deficits that can range from relatively mild to more severe dementia. The prevalence of PSCI ranges from 20% to 80%, following differences in assessment and the diagnostic criteria applied [1]. Despite conspicuous epidemiological relevance, the evaluation of cognitive status is not routine in the acute stroke setting.

The need for an early screening of all patients after stroke has been recommended [2], but in many instances a cognitive assessment is not felt as an immediate clinical priority and a feasible practice early after stroke. Moreover, it is difficult to provide an effective cognitive treatment to all patients, and there is also the potential harm of mislabelling a patient as having a neuropsychological syndrome that can be only transient after the acute event [3]. A comprehensive full neuropsychological battery may still not be appropriate in the acute phase, but brief assessments are possible and may be potentially informative. Up to now, we do not have clear evidence of the cost/benefit ratio of this approach [3] as, in many cases, the National Health Systems cannot easily assure access to effective cognitive treatments, and the adoption of a diagnostic label in the acute phase should be accurately followed up in the chronic phase to account also for spontaneous changes. From the clinical point of view, this is not surprising as the phenomenon of neurocognitive plasticity and spontaneous improvement are well documented in the literature [4], not only in the acute phase, but also later on [5,6].

The early identification of even mild cognitive dysfunction may help guiding the rehabilitation process, identify the most appropriate rehabilitation strategies, and providing a counselling intervention to patients and relatives to overcome the difficulties they may encounter in the future [2].

There is no gold standard for a specific cognitive screening test in the acute stroke patient, but, interestingly, the number of cognitive screening tests is large and continues to grow. The Montreal Cognitive Assessment (MoCA) test was originally developed to identify mild cognitive impairment in aging [7]. MoCA has been widely used also in stroke patients, but its sensitivity and predictive validity in acute stroke settings are reported only in few studies [8], [9], [10]. A more recent instrument, the Oxford Cognitive Screen (OCS) [11], was specifically developed to measure domain-specific cognitive deficits in patients after acute stroke. Only two studies [[12], [13]] have compared the psychometric properties of these two tests and found that, overall, the OCS was more sensitive in detecting impairments than the MoCA; in fact, several subjects with domain-specific impairments in OCS presented a score on MoCA in the normal range. Nevertheless, there is no consensus on which tool is more feasible and informative in detecting cognitive deficits in the acute setting.

The objective of this pilot study was to evaluate the feasibility and acceptability of these two cognitive screening tests in the stroke unit setting, with a focus on patients' psychological status and experiences, administration time, and comparison of patient performances on these two tests.

2. Methods

2.1. Setting

The study was carried out at the stroke units of the Luigi Sacco University Hospital, Milan, Italy and Poliambulanza Hospital, Brescia, Italy. These semi-intensive stroke units admit all the patients who arrive at the respective hospital emergency rooms with an acute cerebrovascular event, except those requiring mechanical ventilation. Both units apply similar care pathways, with patients first admitted to the acute stroke unit and subsequently transferred to the rehabilitation unit if they have persisting impairments and are medically stable.

2.2. Population

All the patients with a diagnosis of ischemic or hemorrhagic stroke or transient ischemic attack (TIA) admitted to the stroke unit of the Luigi Sacco Hospital between January 29, 2020 and February 29, 2020 (27 patients) and to the stroke unit of Poliambulanza Hospital between February 18, 2020 and February 20, 2020 (9 patients) were included in the study.

The study was conducted in accordance with International Conference on Harmonisation Guidelines for Good Clinical Practice, applicable regulations, and guidelines governing clinical-study conduct that have their origin in the Declaration of Helsinki. All clinical data were collected for routine patient care.

The study was planned to recruit a higher number of patients but, due to COVID-19 pandemic, most of the emergency services in Northern Italy were converted into COVID-19 units, and therefore the recruitment of our study was suddenly and prematurely stopped on March 1, 2020.

2.3. Cognitive screening assessments

Within five days of hospitalization, patients were evaluated using the Italian versions of MoCA [14] and OCS [15] by a trained neuropsychologist. MoCA is a pencil-paper test that evaluates nine cognitive abilities limiting the language and grapho-motor impact: visuospatial abilities (clock-drawing task and cube copy), executive functioning (simplified alternating trail making, phonemic fluency task), abstraction (word similarity), language (animal naming, repetition of two syntactically complex sentences), attention (digit span forward and backward), sustained attention and response control (target detection using tapping), calculation, memory (five‐word delayed recall), and orientation (time and place). MoCA global score ranges from 0 to 30 and was computed as suggested by Nasreddine et al. [7]. The Oxford Cognitive Screen (OCS) is a pencil-paper test that evaluates five cognitive abilities: attention (visuospatial, task switching), memory (orientation, recall and recognition), language (picture pointing, picture naming, reading), praxis (imitation sequencing) and numbers (writing calculation). The OCS provides a domain-specific cognitive profile designed for stroke survivors and not a global score and is built to avoid language and neglect impairment influence.

The OCS and MoCA scoring were computed according to Lees et al. [16]; this approach permits assigning a minimum value (i.e., 0) to any incomplete items, hence giving a total score of 0 in the case of entirely incomplete tests [16]. The adoption of this approach allowed us to include all the patients’ results, although with incomplete performance.

A random draw predetermined the order in which the two tests were administered. For each patient, only one examiner had to administer the two tests.

In addition, a structured observation datasheet was developed ad hoc to collect, at the end of each test, all qualitative information available (Fig. 1) on neurological data (i.e., diagnosis, site of the lesion, hemiparesis, aphasia, neglect), duration of test administration, the sequence of administration, and the setting (i.e., bedside or table). This tool allowed us to collect qualitative information about patients’ attitudes, behavior and psychological status (mood and anxiety, aggressivity, impulsivity and fatigue) observed during the testing session. These variables are known to interfere with cognitive performance [17].

Fig. 1.

Structured observation and qualitative information datasheet. The datasheet was developed to collect, at the end of each test, all qualitative information available on neurological data, duration of the test, the sequence of administration, and the setting and includes also different variables which are known to interfere with cognitive performance, such as the patient's attitude and psychological status.

Each item was scored as present/absent by the neuropsychologist who administered the tests based on the clinical judgment during test execution. Finally, a Visuo-Analogical Scale (VAS) was applied to evaluate subjective difficulty perceived by the patient to complete each of the screening tests. Patients were required to rate on a 0–10 scale (0=very easy, 10=very difficult) immediately after completing each test (Fig. 2).

Fig. 2.

Visuo-Analogical Scale (VAS). The scale evaluates subjective difficulty perceived by each patient to complete each of the screening tests (0–10 scoring scale).

2.4. Clinical assessment

Within 5 days of admission, patients underwent an assessment of clinical parameters that included the Cumulative Illness Rating Scale (CIRS; [18]) for the evaluation of comorbidity, the National Institutes of Health Stroke Scale (NIHSS; [19]) for the evaluation of stroke severity at the time of admission and the modified Rankin Scale (mRS) at the time of admission and on discharge from the stroke unit [20].

In addition, the premorbid Clinical Dementia Rating (CDR; [21]) for cognitive impairment, and premorbid global functioning with Activities of Daily Living and Instrumental Activities of Daily Living (ADL/IADL;[22]) were administered to a caregiver.

3. Statistical analysis

Descriptive analyses were performed for age, education and clinical variables. To test the normal distribution of data, we used Shapiro-Wilk tests. A Mann-Whitney Test was used to compare age and education between sexes. T-test was used to check significant differences in the duration of MoCA and OCS. To evaluate the influence of the sequence of administration of the two cognitive screening tests on duration and subjective difficulty, we performed a T-test for each variable (MoCA difficulty, MoCA duration, OCS difficulty, OCS duration). Spearman's coefficients were used to assess the correlation between duration, subjective difficulty, age and education for the MoCA and the OCS. To compare the feasibility of MoCA and OCS (in terms of the number of administrable subtests), we used the Wilcoxon Test.

Furthermore, we explored whether fully, partially and non-completed items were differently distributed within the two tests using a χ² with Yates's continuity correction. Finally, we selected only patients that were able to fully complete both the MoCA and OCS to compare the performance at the two screening batteries. In particular, the total scores computed for the MoCA were considered within the normal range if they were ≥26 [7], adding one extra point for patients with an education ≤12 years as suggested by Gagnon and colleagues [23], while the OCS overall performance was considered below the normal range if there was a deficit in at least one single domain. As a consequence, we obtained a 2 × 2 contingency table that was analysed using the Fisher's Exact Test. Statistical analyses were performed using Jamovi Software version 1.2.16

4. Results

Demographic and clinical data of the sample are reported in detail in Table 1. A statistically significant difference (U = 38.5, p = 0.004) was present in the age between sexes (males mean ages, 67.0 ± 16.0 years, females mean age, 82.4 ± 11.6 years).

Table 1.

Demographic and clinical data.

PATIENTS		N = 34
AGE	Overall group	71.1 ± 16.1
	Males	67.0 ± 16.0
	Females	82.4 ± 11.6
SEX	M/F	25/9
EDUCATION (YEARS)		10.0 ± 4.4
TYPE OF CEREBROVASCULAR EVENT (%):	Ischemic stroke	61.8%
	Hemorrhagic stroke	20.6%
	TIA	17.6%
SIDE OF LESION (%)*:	Right hemisphere	35.7%
	Left hemisphere	50.0%
	Bilateral	14.3%
HANDEDNESS (RIGHT/LEFT)		31/1
HEMIPARESIS (%)		28.1%
APHASIA (%)		15.6%
NEGLECT (%)		9.4%
SETTING (%):	Bed	53.1%
	Table	46.9%
CLINICAL ASSESSMENT		N = 33
MRS		0.7 ± 1
CIRS TOTAL		3.1 ± 2.3
CDR ADMISSION SCORE:	Not available	1 (3.03%)
	CDR=0, n, (%)	24 (72.7%)
	CDR=0.5, n, (%)	4 (12.1%)
	CDR≥1, n, (%)	4 (12.1%)
NIHSS ADMISSION SCORE		4.5 ± 3.6
PREMORBID ADL LOST		0.1 ± 0.7
PREMORBID IADL LOST		1.3 ± 2.2
MRS DISCHARGE SCORE		2.1 ± 1.3

*= 6 missing as they had TIA.

4.1. Tests feasibility and acceptability

Out of the 34 consecutive patients, only 20 patients were able to complete both tests. MoCA was completed by 25 patients, OCS by 21 patients. MoCA was partially administered to 6 patients, while OCS to 11 participants. Only two patients could not perform any part of both tests (see Fig. 3). MoCA was not administered to one additional patient who performed only part of the OCS. The main reasons for not completing the test were: poor vision and confusional state (1), comprehension deficits (1), or comatose state (1).

Fig. 3.

Percentage of patients that completed the full test, a partial version of the test or no section of the test (N = 34).

The feasibility of each MoCA and OCS subtest are reported in Table 2. At the time of administration, most patients appeared euthymic, adequate and collaborative when performing both tests, and we did not observe changes in behavior during the MoCA and OCS administration (Table 3).

Table 2.

Feasibility of each MoCA and OCS subtest.

MOCA

OCS

COGNITIVE DOMAINS	SUBTEST	% ADMINISTRABLE SUBTEST (N)	COGNITIVE DOMAINS	SUBTEST	% ADMINISTRABLE SUBTEST (N)
Visuospatial	Trial Making B	85.29 (29)	Language	Denomination	88.23 (30)
	Cube	82.35 (28)		Semantic	91.18 (31)
	Clock	85.29 (29)		Reading	88.23 (30)
Denomination	Animal Naming	91.18 (31)	Memory	Orientation	94.12 (32)
Attention	Digit Span	88.23 (30)		Recall and Recognition	85.29 (29)
	A Series	88.23 (30)	Number	Writing	91.18 (31)
	Serial 7 Subtraction	88.23 (30)		Calculation	91.18 (31)
Language	Repetition of Sentences	85.29 (29)	Perception	Visual Field	91.18 (31)
	F Fluency	85.29 (29)	Spatial Attention	Heart Cancellation	91.18 (31)
Abstraction	Word Similarity	88.23 (30)	Praxis	Imitation	82.35 (28)
Delayed Recall	Recall	85.29 (29)	Executive Functions	Executive Task	64.70 (22)
Orientation	Orientation	88.23 (30)

Table 3.

Patients’ mood, attitude and behaviour during cognitive assessment.

	MoCA	OCS
	% of patients	% of patients
CURRENT PSYCHOLOGICAL STATUS*
Euthimic	82.35	85.29
Depressed	2.94	2.94
Euphoric	11.76	11.76
Anxious	5.94	2.94
SOCIAL INTERACTION DURING THE TEST*
Adequate	79.41	85.29
Aggressive	0	0
Impulsive	0	0
Restless	14.70	11.76
Apathetic	5.90	5.90
TEST ATTITUDE*
Collaborative	91.17	91.17
Oppositive	0	0
Hasty	11.76	11.76
Needs Solicitations	11.76	14.70
Refuting	2.94	2.94

* a patient can present with more than one status.

A significant difference between the duration of the two tests was observed [t₃₁=−4.48; p<0.001; S-W = 0.149 (OCS: mean ± SD 17.1 ± 6.0 min; MoCA: 14.2 ± 6.1 min)] (Fig. 4). In contrast, the analysis of the VAS scale scores showed that MoCA was perceived as more difficult than OCS [Wilcoxon's test = 168; p<0.003; S-W = 0.037 (MoCA median=4.0, IQR= 5.0; OCS median=3, IQR= 4.0)] (Fig. 4).

Fig. 4.

Duration and subjective difficulty of MoCA and OCS. A significant difference between the durations of the two tests was observed (OCS longer than MoCA). On the VAS scale, MoCA was perceived as more difficult than OCS.

The order of administration did not affect either the durations, or the perceived difficulty of the two tests (OCS duration, t₃₁= −0.631, p = 0.533; VAS OCS, t₂₂ =1.723, p = 0.100; MoCA duration, t₃₁= 0.115, p = 0.909; VAS MoCA, t₂₂= −0.535, p = 0.599).

For each test, correlation analyses showed a direct correlation between the test duration and the perception of test difficulty (MoCA: rho=0.513, p = 0.012, OCS: rho=0.422, p = 0.045). Patient age correlated significantly with the perceived difficulty on the MoCA (rho= 0.745; p<0.001), but not with its duration (p = 0.239). The opposite pattern of results emerged for the OCS as age correlated with test duration (rho=0.435; p = 0.013), but not with its perceived difficulty of it (p = 0.065).

Considering the percentage of administrable subtests, the MoCA and OCS were equally administrable (W = 22.0, p = 0.608) (see Table 2 for details). As a final step, the performance at each single subtest of the OCS and MoCA were classified according to a three-categories scale (fully, partially, non-completed); the χ² analysis showed a significant association between the two screening tools (χ²₃₃=31.8; p<0.001).

4.2. Comparison of test performances

Cognitive performances on MoCA and OCS subtests are reported in Table 4.

Table 4.

Patients' performances on MoCA and OCS subtests.

MoCA				OCS
DOMAINS	ITEM (MAX SCORE)	MEAN ± SD	MEAN PERCENT	DOMAINS	ITEM (MAX SCORE)	MEAN ± SD	MEAN PERCENT
Visuospatial	Trial Making B (1)	0.75 ± 0.43	75.00	Language	Denomination (4)	3.30 ± 0.78	82.50
	Cube (1)	0.40 ± 0.48	40.00		Semantic (3)	3.00 ± 0	100
	Clock (3)	1.75 ± 0.94	58.33		Reading (15)	13.90 ± 1.60	92.67
Denomination	Animal Naming (3)	2.85 ± 0.47	95.00	Memory	Orientation (4)	4.00 ± 0	100
Memory	Repetition (10)	8.80 ± 1.40	88.00		Recall and Recognition (8)	7.00 ± 1.09	87.50
Attention	Digit Span (2)	1.40 ± 0.73	70.00	Number	Writing (3)	2.90 ± 0.43	96.70
	A Series (3)	0.80 ± 0.40	26.67		Calculation (4)	3.90 ± 0.43	97.50
	Serial 7 Subtraction (3)	2.75 ± 0.53	91.67	Perception	Visual Field (4)	3.85 ± 0.47	96.25
Language	Repetition of Sentences (2)	1.25 ± 0.76	62.50	Spatial Attention	Heart Cancellation (50)	44.85 ± 4.67	89.70
	Phonemic Fluency (1)	0.40 ± 0.48	40.00	Praxis	Imitation (12)	9.45 ± 2.51	78.75
Abstraction	Word Similarity (2)	1.20 ± 0.74	60.00	Executive Functions	Executive Task (12)	1.10 ± 2.82	9.17
Delayed Recall	Recall (5)	1.55 ± 1.80	31.00
Orientation	Orientation (6)	5.65 ± 0.72	94.17

Among the 20 patients who had both the MoCA and the OCS entirely completed, only 2 patients did not show any impairment. Seventeen patients (85%) were impaired in at least one cognitive domain of the OCS, and 15 patients (75%) were below the MoCA cut-off. Of the 5 patients with MoCA score above the cut-off, 3 were impaired in at least one of the cognitive domains of the OCS. Interestingly, the Fisher's Exact Test did not reveal any association between the patients’ classification made according to the two screening tools (Fisher's Exact Test=7.99; p = 0.140, see Table 5). Only 1 out of the 6 TIA patients obtained a score within the normal range at both tests, while 3 TIA patients had more than one domain compromised on OCS.

Table 5.

Comparison of MoCA and OCS performances (2 × 2 contingency table).

	MOCA <26	MOCA ≥26	TOTAL
OCS ≥1 IMPAIRED DOMAIN	15 (75%)	2 (10%)	17 (85%)
OCS WITHOUT IMPAIRED DOMAIN	1 (5%)	2 (10%)	3 (15%)
TOTAL	16 (80%)	4 (20%)	20 (100%)

5. Discussion

This preliminary report aimed to compare MoCA and OCS feasibility and acceptability in the acute stroke setting. Overall, our study did not show a superiority of the OCS over the widely used MoCA and suggests the need for further validation of the MoCA and OCS in larger samples.

A previous study by Demeyere and colleagues on a sample of 200 patients found that OCS was more sensitive in detecting cognitive impairment after stroke than MoCA test as OCS detects important deficits not directly assessed in the MoCA, such as neglect or apraxia [12]. In a following longitudinal study, Demeyere and colleagues compared MoCA and OCS in the assessment of cognitive disorders in a sample of 821 acute stroke patients and found a good agreement between the two tests[13].

The main novelty of our work is the assessment of several feasibility and acceptability variables that are usually not recorded in clinical practice (i.e. duration, perceived difficulties, patient's behavior during the test). Moreover, the population and the timing of assessment selected by Demeyere was different, as they applied other inclusion/exclusion criteria, enrolling patients within 3 weeks of a confirmed ischemic or hemorrhagic stroke (and not only focused on the hyperacute setting) [12].

There is a need for time-efficient screening tools to be used in this specific patient population with a high prevalence of physical, perceptual, mood, and cognitive problems. While OCS was explicitly designed for an acute stroke population, MoCA was created as a screening test in the field of dementia. Based on these considerations, clinicians may assume that the former is more feasible and informative than the latter for patients with stroke. Our very preliminary experience however does not support this view.

It is worth noting that the development of new clinical instruments for cognitive testing in stroke must also consider feasibility, acceptability, and opportunity cost [24] beyond the specific purpose of the test itself. For example, when in the validation study, the testing is either partially or entirely incomplete, it is common to exclude data from these patients. This approach can lead to biased results [16]. For this reason, we decided to include all the data in our analysis and assign a minimum value to each incomplete item, hence giving a total score of 0 where testing was entirely incomplete (the most inclusive approach suggested by [16]).

In our sample, most participants needed assistance to complete tests and even our “short” assessments had substantial rates of non-completion as none of the items from the two screening tests was doable by all of 34 patients. This result is in line with the well-known difficulty in evaluating cognitive functions in acute stroke patients. Moreover, by definition, a short administration time is essential for a battery of tests designed to “quickly” screen for cognitive disorders. The OCS administration time was on average adequate for a screening tool (about 20 min) but it was longer than that of the MoCA. Finally, none of the patients explicitly reported fatigue in completing the test. Indeed, OCS was perceived as less demanding, and this aspect may impact on the acceptability of the tool by the patient.

Moving to the feasibility dimension, our results partially extend the growing body of empirical evidence about cognitive testing in stroke [24]. Completion rates of 68%–80% for MoCA have been reported in acute stroke [25,26], and our data are in line with the literature and add a little piece of empirical evidence for what concerns the feasibility of the OCS which, so far, was tested only by the authors who developed the test.

No difference was observed in the psychological status and social behaviour of the patient during both test administration and most of the patients were euthymic, adequate and collaborative. These observations support feasibility of both screening tools in the stroke unit setting. Unfortunately, it was not possible to directly compare the cognitive domains assessed by the two tests as the label for a cognitive domain is not consistent across the instruments (e.g., Orientation subtest in the OCS is part of “Memory” domain, while in the MoCA is a separate domain). Therefore, similar labels might tap into different cognitive domains or different labels might measure the same domain [27].

Our results also suggested high agreement rates between the cognitive screening measures, but the qualitative and quantitative aspects of the two tests could make them appropriate for different purposes. The MoCA global score may serve as an initial screening index after acute stroke in large-scale clinical studies, predict long-term cognitive and functional outcome, and guide further in-depth neuropsychological assessments. On the other hand, the OCS could be more informative through its visual snapshot cognitive profile to plan individualized rehabilitation programs in patients with acute stroke.

There are limitations in our study. First, the modest sample size that was significantly limited by unforeseen circumstances compared to our original plan. We did not pre-specify a sample size because one of the metrics of interest was the feasibility of recruitment to a cognitive study over a fixed time-period (i.e. three months).

This issue did not allow us to run correlation analyses with side of the lesion, stroke severity and cognitive subdomains, which would have represented an added value for the study.

Second, the lack of a longitudinal follow-up with a more comprehensive neuropsychological and clinical assessment that may provide further information regarding a possible difference between tests in their predictive values.

The value of our approach relies on the collection of several qualitative information related to the patients’ experience and setting (time, subjective difficulty, place of assessment). Indeed, the structured observational methodology allowed us to evaluate the patient's status during the administration of the screening tests; these variables are not part of a routine cognitive evaluation but may be informative from the clinical point of view.

6. Conclusion

In conclusion, MoCA and OCS appear to be useful brief tools to assess cognition in patients after stroke. Additional comparative studies on a larger sample, including correlational analyses of cognitive data with stroke severity and side of the lesion and a follow-up assessment will be conducted as soon as the pandemic emergency in Italy will be contained.

Declaration of Competing Interest

There is no conflict of interest to be disclosed by any of the authors