Who should undergo a comprehensive cognitive assessment after a stroke?

Abstract

Objective

To validate the ability of a specifically developed cognitive risk score to identify patients at risk of poststroke neurocognitive disorders (NCDs) who are eligible for a comprehensive cognitive assessment.

Methods

After assessing 404 patients (infarct 91.3%) in the Groupe de Réflexion pour l'Evaluation Cognitive VASCulaire (GRECogVASC) cross-sectional study with the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network battery 6 months after stroke, we used multivariable logistic regression and bootstrap analyses to determine factors associated with NCDs. Independent, internally validated factors were included in a cognitive risk score.

Results

Cognitive impairment was present in 170 of the 320 patients with a Rankin Scale score ≥1. The backward logistic regression selected 4 factors (≥73% of the permutations): NIH Stroke Scale score on admission ≥7 (odds ratio [OR] 2.73, 95% confidence interval [CI] 1.29–4.3, p = 0.005), multiple strokes (OR 3.78, 95% CI 1.6–8, p = 0.002), adjusted Mini-Mental State Examination (MMSEadj) score ≤27 (OR 6.69, 95% CI 3.9–11.6, p = 0.0001), and Fazekas score ≥2 (OR 2.34, 95% CI 1.3–4.2, p = 0.004). The cognitive risk score computed with these 4 factors provided good calibration, discrimination (overoptimism-corrected C = 0.793), and goodness of fit (Hosmer-Lemeshow test p = 0.99). A combination of Rankin Scale score ≥1, cognitive risk score ≥1, and MMSEadj score ≥21 selected 230 (56.9%) of the 404 patients for a comprehensive assessment. This procedure yielded good sensitivity (96.5%) and moderate specificity (43%; positive predictive value 0.66, negative predictive value 0.91) and was more accurate (p ≤ 0.03 for all) than the sole use of screening tests (MMSE or Montréal Cognitive Assessment).

Conclusion

The GRECogVASC cognitive risk score comprises 4 easily documented factors; this procedure helps to identify patients at risk of poststroke NCDs who must therefore undergo a comprehensive assessment.

ClinicalTrials.gov identifier:

NCT01339195.

Poststroke neurocognitive disorders (NCDs) are major outcome events observed in about half of survivors.^1,2 These disorders have a marked effect on functional prognosis, institutionalization,^3–5 and recurrence of a major vascular event.^6,7 The administration of a comprehensive neuropsychological battery is the gold standard for the diagnosis of an NCD. Given that a comprehensive cognitive assessment is not feasible in all assessable stroke survivors, a simpler test such as the Mini-Mental State Examination (MMSE)⁸ and the Montréal Cognitive Assessment (MoCA)⁹ is frequently used as a first-line screen. However, these tests have only moderate to good sensitivity,^10,11 which thus underestimates the impairment in a significant proportion of affected patients. In the present study, we examined a previously unexplored strategy based on the identification of patients at risk of poststroke NCDs who require referral for a comprehensive assessment. Factors associated with poststroke NCDs have been assessed for dementia¹² and (to a lesser extent) mild NCDs,² which account for about two-thirds of cases of poststroke cognitive impairment.^1,2 Furthermore, the minimal set of factors for selecting patients at risk of full-spectrum NCDs (i.e., ranging from mild NCD to dementia) has not previously been examined.

The objective of the present study was to develop a multivariable prediction model for the risk of NCDs 6 months after a stroke as a tool for helping clinicians to select patients who require a comprehensive cognitive assessment. Our approach is based on clinical and radiological datasets that are commonly available in routine clinical practice and was validated in the Poststroke Cognitive Impairment and Dementia (GRECogVASC) cohort.^1,13

Methods

Population

The design and main results of the cross-sectional GRECogVASC study have been reported elsewhere.^1,13 Briefly, the eligible population consisted of French-speaking patients between 40 and 80 years of age who were hospitalized for acute cerebral infarct or hemorrhage with initial positive imaging results, a reliable informant, and no previously diagnosed conditions affecting cognition (except for previous stroke). The main exclusion criteria were as follows: known conditions (other than stroke) affecting cognition (chronic alcoholism; substance addiction; liver, kidney, or respiratory failure; and paraneoplastic syndrome treatments affecting cognition other than stable dosage levels of an anxiolytic or a serotoninergic antidepressant), previously diagnosed psychiatric conditions (schizophrenia and major psychiatric disorders requiring hospitalization for >2 days in a specialized setting), neurological conditions (mental retardation, dementia, epilepsy, severe traumatic brain injury, Parkinson disease, multiple sclerosis, a brain tumor, or brain radiotherapy), conditions precluding cognitive assessment illiteracy, severe sensory or motor impairments, alertness disorder, comorbid conditions associated with a life expectancy <2 years, contraindication to MRI, pregnancy, legal guardianship, and lack of written informed consent. Aphasia, hemineglect, and prior stroke were not exclusion criteria. A total of 404 consecutive patients were included.¹

Clinical and neuropsychological examinations and MRI were performed 6 months after the index stroke according to a protocol previously detailed.¹ Neuropsychological performance was assessed with the French adaptation of the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network comprehensive battery,^13,14 which constituted the gold standard. It provided z scores for 5 cognitive domains (action speed, executive functions, episodic memory, language, and visuoconstructive abilities).^1,13 Both the MMSE and MoCA were used as a screening test and were interpreted after adjustment for demographic factors according to GRECogVASC norms.¹⁵ The raw MMSE score was adjusted as a function of educational level (MMSEadj) (educational level 1: raw MMSE score + 1; level 2: raw MMSE score; level 3: raw MMSE score − 1) because this approach was found to control for demographic factors in a group of 1,003 healthy controls (age p = 0.3, educational level p = 0.4). In addition, we defined severe impairment on a screening test as the cutoff score always associated with impairment on comprehensive cognitive assessment. This corresponded to an MMSEadj score ≤20 or MoCA score ≤14 as computed from both the GRECogVASC database and the AmiensCog database using an independent sample of 650 comprehensive assessments.^1,15–17 The Rankin Scale was graded using a structured interview including difficulties in instrumental activities of daily living.¹⁸

Mild and major NCDs were diagnosed according to the International Society for Vascular Behavioral and Cognitive Disorders criteria¹⁹ with the optimized criterion for cognitive impairment^2,20 and norms acquired in 1,003 healthy controls.¹⁵

Statistical analyses

Because our objective was to identify patients at risk of cognitive impairment who require referral for a comprehensive cognitive assessment, we first selected patients with a Rankin Scale score ≥1; a comprehensive assessment was considered to be futile in patients who regained all prestroke activities without any concerns (according to both the patient and the informant), as assessed with the Rankin structured interview.¹⁸ In addition, cognitive concern as evidenced by Rankin structured interview is needed to fit the criteria of NCDs.¹⁹ Thus, the first factor used to select patients at risk was the Rankin Scale score (0 vs ≥1).

In the 320 patients with a Rankin Scale score ≥1, we examined the association of NCDs with data that were easily and reliably available in routine clinical practice and with regard to the following characteristics: demographic factors (age, sex, educational level, and characteristics of the living environment), vascular risk factors defined with common criteria¹ (arterial hypertension, hypercholesterolemia, statin use, diabetes mellitus, overweight, metabolic syndrome, a family history of stroke, current and past smoking, current and past alcohol abuse, atrial fibrillation, prior myocardial infarct, previous stroke and previous ischemic stroke, prior migraine with and without aura, prior sleep apnea syndrome, prior depression, and the prestroke Informant Questionnaire on Cognitive Decline in the Elderly [IQCODE] score), the characteristics of the index stroke (the NIH Stroke Scale [NIHSS] on admission and the stroke subtype and cause), events in the 6 months after the stroke (the recurrence of stroke or the occurrence of epilepsy and depression), and the MRI characteristics (infarct vs hemorrhage, presence of previous stroke lesions, index stroke with multiple lesions, hemispheric vs posterior fossa stroke, left hemispheric stroke,²¹ presence of microbleeds,²² deep white matter abnormalities,²³ and the sum of the left and right hippocampal atrophy scores on the Scheltens scale).²⁴ Index stroke with multiple lesions was defined as lesions that were not confluent, whatever their territory.

Selection of factors in the model

The associations of various factors with cognitive status were first examined in bivariable analyses (using the Student t test for continuous variables and the Fisher exact test or a χ² test for other variables). Factors with a value of p < 0.2 were fed into a backward logistic regression. To examine whether selection criteria were influenced by age, the interaction between selected factors and age was also analyzed. Because the study objective was to provide predictive factors for both cerebral infarct and hemorrhage, causes of stroke were not considered in these analyses. Furthermore, given that models derived from multivariable regression analyses are likely to be overly optimistic, we validated our model internally by applying bootstrapping techniques²⁵ with 1,000 permutations. Only factors selected in at least 500 permutations were retained in the multivariable model. To facilitate the setup of a global risk score, continuous scores (age, NIHSS score, MMSEadj score, Fazekas score, and the sum of left and right hippocampal atrophy scores) were dichotomized with receiver operating characteristic curve analysis; cognitive status was used as the external criterion (normal vs impaired), and the cutoff score was determined with the Youden index.

Elaboration and testing of a cognitive risk score

On the basis of the final multivariable model, we used bootstrap procedures (1,000 permutations) to estimate the shrinkage factor, the bias-corrected calibration plot, and the C statistic corrected for overoptimism.²⁵ Shrunk β regression coefficients improve predictions for future cohorts,²⁵ so they were used to determine the weight of factors entered into the cognitive risk score. The weight of each variable was determined with a linear transformation as follows: divide all shrunk regression coefficient by the lowest one and round to the nearest integer. The level of performance of the cognitive risk score was analyzed by the use of both raw and bias-corrected calibration plots, the C statistic corrected for overoptimism, and the Hosmer-Lemeshow test.²⁵ Visual inspection of the calibration plots and Hosmer-Lemeshow test indicated whether the observed cases of cognitive impairment matched the cases predicted by the cognitive risk score. Performance of the procedure based on risk score was compared with that provided by usual screening tests (i.e., MMSE and MoCA) using the McNemar test.

With regard to sample size, 170 patients with cognitive impairment are sufficient to provide stable estimates in a model with 17 factors (given 10 outcomes per factor as a rule of thumb).²⁶

The report of the study follows the Strengthening the Reporting of Observational Studies in Epidemiology statement guidelines.

Standard protocol approvals, registrations, and patient consents

The clinical trial identifier of the GRECogVASC study is NCT01339195. The study was performed in accordance with institutional guidelines, implying that informed consent was obtained from all participants, and was approved by the regional investigational review board (Comité de Protection de Personnes Nord-Ouest II, Amiens, France; reference 2010/25).

Data availability

Deidentified participant data used in this specific study can be shared on request for pooled studies devoted to the same objective as the present one.

Results

A Rankin Scale score ≥1 was observed in 320 (79%) patients, whose demographic and clinical characteristics were typical of a hospital-based stroke population (table 1). Stroke was due mainly to infarct 40 (14%) strokes were treated at the acute phase by IV thrombolysis. NCDs were observed in 170 patients (53.1%); 129 patients (75.9%) had a mild NCD, and 41 (24.1%) had a major NCD.

Table 1.

Demographic, clinical, and imaging characteristics of the GRECogVASC study group and results of a bivariable analysis as a function of cognitive status

Selection of factors in the model

The bivariable analysis showed that cognitive impairment was associated with age, education level, arterial hypertension, prestroke IQCODE score, NIHSS score on admission, MMSEadj score, MoCA score, multiple strokes, left hemispheric stroke, Fazekas score, the presence of microbleeds, and hippocampal atrophy (table 1). Prestroke IQCODE score and the poststroke interval were not included in the multivariable analysis because their effect was very weak (area under the curve: IQCODE 0.563, 95% confidence interval [CI] 0.522–0.653; time interval 0.559, 95% CI 0.496–0.622) and thus prevented the determination of a reliable cutoff. The MMSEadj score was preferred to the MoCA score because the 2 screening tests have a similar sensitivity^10,11 and because the MMSE is easier to adjust for demographic factors. The following factors were fed into the backward selection logistic regression: age (cutoff >70 years), female sex, educational level (≤8 years, i.e., primary level), arterial hypertension, diabetes mellitus, hypercholesterolemia, previous ischemic stroke, NIHSS score on admission (cutoff ≥7), MMSEadj score (cutoff ≤27), multiple strokes, left hemispheric stroke, Fazekas score (cutoff ≥2, defined as beginning confluence of foci²³), the presence of microbleeds, and hippocampal atrophy (cutoff ≥3). The backward logistic regression selected 4 factors (table 2): NIHSS score on admission ≥7, multiple strokes, MMSEadj score ≤27, and a Fazekas score ≥2. The bootstrap procedure selected these 4 factors in at least 73% of permutations (table 2). Age did not interact with these factors (p > 0.48, all).

Table 2.

Variables considered in multivariable analysis of poststroke cognitive impairment

Elaboration and testing of a cognitive risk score

With the use of these 4 factors, raw and shrunk β coefficients (overall shrinkage factor 0.9621) were computed (table 2). After the standardization of shrunk coefficients, the cognitive risk score was computed as follows: NIHSS score on admission ≥7 (present = 1; absent = 0) + multiple strokes (present = 1; absent = 0) + MMSEadj score ≤27 (present = 2; absent = 0) + Fazekas score ≥2 (present = 1; absent = 0). Age did not interact with the cognitive risk score (p = 0.5).

The plots showed a good level of calibration even after correction for bias (figure 1). The discriminative ability of the score was good (raw C statistic 0.7933, 95% CI 0.745–0.842, overoptimism-corrected C statistic 0.7932). Goodness of fit (Hosmer-Lemeshow test: χ2 = 1.06, df = 8, p = 0.99) was very good. A cutoff was selected with the objective of optimizing the sensitivity of the screening procedure; this yielded a cognitive risk score ≥1 as the cutoff (sensitivity 96.5%), whereas a cutoff score ≥2 would have resulted in a sensitivity of 80%.

Figure 1. Calibration plot (A) with raw scores and (B) after bias correction.

Note that scores 4 and 5 were pooled because only 4 patients had a cognitive risk score of 5. NCD = neurocognitive disorder.

Performance of the procedure based on cognitive risk score

Performance of the procedure based on cognitive risk score was analyzed by applying the 3 factors to the full set of 404 patients (figure 2): modified Rankin Scale score (0 vs ≥1), cognitive risk score (0 vs ≥1), and severe impairment on screening test (i.e., MMSEadj score ≤20 or MoCA score ≤14). Modified Rankin Scale score ≥1, cognitive risk score ≥1, and MMSEadj score ≥21 selected 230 (56.9%) of the 404 patients, 144 (62.6%) of whom were cognitively impaired according to the comprehensive battery. This procedure would have underestimated cognitive impairment in 6 of the 320 patients (sensitivity 96.5%) and would have indicated the need for a comprehensive assessment in 86 individual with normal cognitive status (specificity 43%, positive predictive value [PPV] 0.66, negative predictive value [NPV] 0.91).

Figure 2. Flowchart of the screening procedure for selecting patients for a comprehensive assessment.

MMSEadj = adjusted Mini-Mental State Examination.

Performance of the present procedure was compared to that based on usual screening tests. To obtain a sensitivity of 96% with the usual screening tests would require the use of a cutoff of ≤29 for both raw and adjusted MMSE scores and ≤27 for the MoCA score. The use of these cutoffs would be associated with a high number of selected patients (i.e., lower specificity): MMSEadj score, n = 267 of 320 patients (specificity 29%, PPV 0.6, NPV 0.83); raw MMSE score, n = 280 of 320 patients (specificity 22%, PPV 0.58, NPV 0.83); and MoCA score, n = 264 of 320 patients (specificity 27%, PPV 0.6, NPV 0.87). This would lead to the selection of a higher number of patients than the present risk score would (MMSEadj p = 0.03, raw MMSE p = 0.0001, MoCA p = 0.008).

Discussion

We developed a predictive model for poststroke cognitive impairment based on 4 factors: NIHSS score on admission (≥7), multiple strokes, MMSEadj score (≤27), and Fazekas score (≥2). This risk score differentiates well between stroke patients at risk of cognitive impairment and those not at risk. When applied together with a modified Rankin Scale score and severe impairment in a screening test, the risk score selects 56.9% of patients in the original cohort for assessment with a comprehensive battery. It has good sensitivity (96%), which constitutes a key criterion for high-quality screening. The risk score selects a significantly lower proportion of patients than the usual procedure based on screening tests does.

The risk score is based on 4 factors that are easily collected in routine clinical practice, using data from the acute stage (the NIHSS score), MRI (the presence of multiple stroke lesions and the deep white matter Fazekas score), and the follow-up assessment (MMSEadj). The value of these factors is confirmed by previous reports of their association with poststroke mild NCDs^27–36 and dementia.¹² Other factors associated with cognitive status in a bivariable analysis were not selected for the multivariable analysis. In particular, a high frequency of both mild³⁶ and major¹² NCDs in cerebral hemorrhage was not observed. The internal validation (with a bootstrap analysis) and the large sample size of the study indicate that the nonselected factors are not independent. Nevertheless, these findings must be validated in an external sample with different characteristics.

Although the present cognitive risk score performs better than the usual procedure for selecting patients for a comprehensive cognitive assessment, several considerations and limitations must be taken into account. First, some factors were deliberately not included in the GRECogVASC cohort such as previously diagnosed conditions affecting cognitive abilities. Thus, the use of the present cognitive risk score in clinical practice requires these factors to be taken into account, because they independently contribute to cognitive impairment. This may be especially frequent in very old stroke patients, and this would require a specific study. Second, the requirement of optimal sensitivity for a screening strategy leads to the selection of patients with a cognitive risk score ≥1, which accounted for 57% of the patients in the cohort. A sensitivity of 80% (provided by a cognitive risk cutoff score ≥2) was judged to be too low. Although 57% of the patients is a large proportion, it remains appropriate when considering that poststroke NCD is observed in about half of assessable survivors.¹ Use of the present cognitive risk score in clinical practice may increase the proportion of selected patients, because the mean age and severity of stroke are higher than those observed in the present cohort.³⁷ However, this increase might be counterbalanced by lower MMSE scores in more severe strokes. In particular, this criterion excludes patients with vigilance disorders, inability to sit, major sensory impairments (e.g., blindness and deafness), major motor impairments, mutism, and aphasia from a comprehensive assessment. Third, the cognitive risk score was designed to predict cognitive risk 6 months after stroke. Thus, its validity during the acute stage of stroke and in the longer term has not been examined. Fourth, we excluded patients without acute lesions on initial imaging. Thus, our selection criteria cannot be used in patients without imaging and without stroke lesion. This limitation is counterbalanced by evidence indicating that the presence of a stroke lesion is a major determinant of poststroke cognitive impairment.^38,39 Fifth, our selection criteria included a modified Rankin Scale score ≥1 and severe impairment on a screening test, as well as the cognitive risk score. In clinical practice, we consider that comprehensive assessment is futile in patients who fully recover their prestroke status without any problems. However, this supposes that the Rankin Scale score is graded during a structured interview with a reliable informant (probing cognitive complaints and behavioral changes), because this procedure was found to be sensitive to cognitive impairment.¹⁸ However, a few patients with a Rankin Scale score of 0 (such as those requiring the certification of professional skills) require a comprehensive assessment. In the same vein, we consider that it is of no value to perform a comprehensive assessment in patients with severe impairment on screening tests, because this result is always associated with cognitive impairment in a battery of tests. We defined our cutoff with reference to large French databases,^1,15,17 and the value may vary from one country to another. Furthermore, a few patients with mildly severe impairment on screening tests may require a comprehensive assessment to determine the cognitive profile.

Our present results show that several simple items of clinical and neuroimaging data (gathered easily during routine clinical practice) predict the risk of cognitive impairment and thus enable the selection of patients requiring a comprehensive cognitive assessment. Future research should examine the validity of this poststroke cognitive risk score in an external population.

Glossary

CI

confidence interval

GRECogVASC

Poststroke Cognitive Impairment and Dementia

IQCODE

Informant Questionnaire on Cognitive Decline in the Elderly

MMSE

Mini-Mental State Examination

MMSEadj

adjusted Mini-Mental State Examination

MoCA

Montréal Cognitive Assessment

NCD

neurocognitive disorder

NIHSS

NIH Stroke Scale

NPV

negative predictive value

PPV

positive predictive value

Author contributions

AI = analysis or interpretation of the data; DC = design or conceptualization of the study; DR = drafting or revising the manuscript for intellectual content; MRDA = major role in the acquisition of data. Olivier Godefroy, MD, PhD: DC, AI, DR. Hugo Yaïche, MD: AI, DR. Hervé Taillia, MD: DC, DR. Flavie Bompaire, MD: MRDA, DR. Claudine Nédélec-Ciceri, MD: DC, DR. Camille Bonnin, PhD: MRDA, DR. Jérôme Varvat, MD: DR. Françoise Vincent-Grangette: MRDA, DR. Momar Diouf: AI, DR. Jean-Louis Mas, MD: DC, DR. Sandrine Canaple, MD: DR. Chantal Lamy, MD: DR. Audrey Arnoux, MD: DR. Claire Leclercq, MD: DC, DR. Sophie Tasseel-Ponche, MD: DR. Martine Roussel, PhD: DC, AI, DR. Mélanie Barbay, MD: AI, DR.

Study funding

The study has been funded by Amiens University Hospital and by a grant from the French Ministry of Health (DGOS R1/2013/144).

Disclosure

O. Godefroy has served on scientific advisory boards (Novartis and AstraZeneca) and received funding for travel and meetings from Novartis, Lilly, Genzyme, AstraZeneca, Biogen, Teva, Pfizer, CSL-Behring, GSK, Boehringer-Ingelheim, Ipsen, Covidien, and Bristol-Myers Squibb. H. Yaïche, H. Taillia, F. Bompaire, C. Nédélec-Ciceri, C. Bonnin, J. Varvat, F. Vincent-Grangette, and M. Diouf report no disclosures relevant to the manuscript. J. Mas has served on scientific advisory boards (Bayer, Boehringer-Ingelheim, Bristol-Myers-Squibb, Pfizer). S. Canaple has received travel and meeting support from Boehringer-Ingelheim, Bayer, Biogen, and Bristol-Myers Squibb. C. Lamy has received travel and meeting support from Bayer, Biogen, Bristol-Myers Squibb, Genzyme, Sanofi, Medtronic, St. Jude Medical France, Covidien, PharmaDom, and Pfizer. A. Arnoux has received travel and meeting support from Boehringer-Ingelheim, Biogen, Teva, and Pfizer. C. Leclercq reports no disclosures relevant to the manuscript. S. Tasseel-Ponche has received travel and meeting support from Allergan, DJO, Icomed, Ipsen Pharma, LFB biomedicaments, and Merz. M. Roussel and M. Barbay report no disclosures relevant to the manuscript. Go to Neurology.org/N for full disclosures.

Publication history

Received by Neurology December 19, 2017. Accepted in final form August 13, 2018.