The clinical value of the patient-reported multiple sclerosis neuropsychological screening questionnaire

Abstract

Background:

Cognitive problems are difficult to identify in patients with multiple sclerosis (MS).

Objective:

To investigate the clinical applicability of the patient-reported MS Neuropsychological Screening Questionnaire (MSNQ-P).

Methods:

Cut-off scores were determined to differentiate between cognitively impaired (n = 90), mildly cognitively impaired (n = 115), and cognitively preserved (n = 147) MS patients using receiver operating characteristic analyses.

Results:

We could not define specific and sensitive cut-off scores. Higher scores (≥27) did indicate cognitive impairment. Among patients with a higher education, lower scores (<12) indicated intact cognition.

Conclusion:

Certain scores can indicate intact or impaired cognitive function. Still, MSNQ-P scores should be interpreted with caution.

Introduction

Cognitive problems are prevalent and disabling in patients with multiple sclerosis (MS).¹ Early detection is therefore highly relevant. However, cognitive problems are difficult to detect during routine neurological care due to their subtle nature and weak relation with physical disabilities.¹ Neuropsychological evaluation is a useful tool to assess cognitive problems, and concise neuropsychological batteries are available for MS patients (e.g. Rao’s Brief Repeatable Neuropsychological Battery (BRB-N)² and the Brief International Cognitive Assessment for MS (BICAMS)).³ However, it is not feasible for all healthcare facilities to routinely perform neuropsychological evaluations during neurological care visits. In order to identify patients who may benefit most from neuropsychological evaluation, it is essential to know the clinical value of patient-reported cognitive complaints.

A brief self-report questionnaire to identify cognitive and neurobehavioral problems is the MS Neuro-psychological Screening Questionnaire (MSNQ).⁴ The informant version of the MSNQ (MSNQ-I) is considered a reliable cognitive screening tool,^4–7 but informants do not generally accompany MS patients at routine neurological care visits. The clinical value of the patient-reported MSNQ (MSNQ-P) varied across studies,^4–7 which may be due to small sample sizes or differences in cognitive classification methods. Importantly, psychological factors seem to play an important role in patient-reported cognitive complaints and should therefore be taken into account.^4–7

This study aimed to extend on previous work by investigating the clinical applicability of the MSNQ-P.

Materials and methods

Participants

We selected 352 patients from the Amsterdam MS cohort (65.9% female, age: 50.7 ± 11.8 years, disease duration: 16.8 ± 9.1 years and phenotype: clinically isolated syndrome (6.0%), relapsing–remitting (58.5%), secondary progressive (20.2%), primary progressive (10.5%) or unknown (4.8%)). The patients (1) completed the MSNQ-P (≤1 missing item), (2) had scores on all cognitive domains of the neuropsychological examination and (3) their visit was not included in our previous study⁶ to ensure independence between studies. A total of 88 matched healthy controls (HCs) were also included.

Outcomes

The MSNQ-P measured subjective cognitive complaints⁴ and is validated in a Dutch MS population.⁶ Scores range from 0 to 60 and higher scores indicate more cognitive complaints. The Hospital Anxiety and Depression Scale measured anxiety and depression,⁸ the Checklist Individual Strength-20-R measured fatigue,⁹ and the Expanded Disability Status Scale indicated neurological disability. An extended version of the BRB-N measured cognitive functioning,² including verbal memory, visuospatial memory, information processing speed, attention, working memory, executive function and verbal fluency. Scores were corrected for age, education and gender and transformed into domain-specific z-scores based on HC scores. Patients were classified as (1) cognitively impaired (CI; ≥2 SDs below HCs on ≥2 domains), (2) mildly cognitively impaired (MCI; ≥1.5 SDs below HCs on ≥2 domains and not CI or ≥2 SDs below HCs on 1 domain and <1.5 SDs on the other domains) and (3) cognitively preserved (CP; remainder).

Statistical analyses

We performed receiver operating characteristic (ROC) analyses and Bayesian statistics to determine the optimal cut-off scores (i.e. the highest value for sensitivity and specificity combined) and clinically relevant score ranges. Logistic regression analyses were performed to correct for depression, anxiety and fatigue. To identify relevant predictors of MSNQ-P scores, two backward linear regression models were performed including (1) cognition, depression, anxiety, fatigue, age, disease duration, education and neurological disability (complete data were available for 245 patients) and (2) domain-specific cognitive z-scores (p < 0.05).

Results

The optimal cut-off scores and psychometric properties are presented in Table 1. The optimal cut-off score to screen for CI or both MCI and CI resulted in an area under the curve (AUC) of 0.61 and 0.64, respectively. The AUCs did not increase when correcting for depression (0.63 and 0.66, respectively), anxiety (0.62 and 0.63, respectively), or fatigue (0.59 and 0.59, respectively). The AUCs also did not increase when neurobehavioral MSNQ-P items (13–15) were excluded (0.61 and 0.63, respectively). Since MSNQ-P scores were related to education, we performed stratified analyses for education (see Supplementary Table 1). The AUCs were higher among patients with a higher education (0.68 and 0.69, respectively), compared to a middle (0.50 and 0.55, respectively) or lower education (0.58 and 0.56, respectively). In patients with a higher education, the optimal cut-off score to screen for CI and for both MCI and CI was 12, which corresponded to a high sensitivity (0.87 and 0.85, respectively) and negative predictive value (0.93 and 0.81, respectively), yet a relatively lower specificity (0.40 and 0.52, respectively) and positive predictive value (0.25 and 0.59, respectively). Figure 1 shows the distribution of MSNQ-P scores and clinically relevant score ranges.

Table 1.

MSNQ-P cut-off scores and psychometric properties.

	Cut-offa	Sensitivity	Specificity	PPV	NPV	Accuracyb	AUC
CI vs MCI and CP	18	0.66	0.51	0.31	0.81	0.55	0.61*
CI and MCI vs CP	12	0.82	0.40	0.66	0.62	0.65	0.64*

CI: cognitively impaired; MCI: mildly cognitively impaired; CP: cognitively preserved; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.

^aOptimal cut-off score on the patient-reported Multiple Sclerosis Neuropsychological Questionnaire (MSNQ-P) to detect the cognitively impaired patient group (either CI or a combination of both CI and MCI patients).

^bProportion of patients correctly classified as cognitively impaired and cognitively preserved.

^*Significantly different from 0.50 (p < 0.01).

Figure 1.

Distribution of MSNQ-P scores across the cognitive groups. The bold lines represent the means and SDs (*p < 0.001). Of all patients, 41.8% were categorized as CP (n = 147), 32.7% as MCI (n = 115), and 25.6% as CI (n = 90). (a) The CI and MCI groups both had significantly higher MSNQ-P scores than the CP group. (b) The CI and MCI groups combined had significantly higher MSNQ-P scores than the CP group. The dotted lines indicate the clinically relevant score ranges (defined as a positive predictive value (PPV) or negative predictive value (NPV) of at least 0.70). The upper line indicates that 71% of the patients with scores of 27 or higher were MCI or CI patients. The lower line indicates that 70% of the patients with scores of 5 or lower were CP patients.

CP: cognitively preserved; MCI: mildly cognitively impaired; CI: cognitively impaired.

The final backward regression model showed that cognition (MCI vs CP: beta = 0.13, p = 0.020; CI vs CP: beta = 0.12, p = 0.045), depression (beta = 0.20, p = 0.008), anxiety (beta = 0.19, p = 0.006), fatigue (beta = 0.39, p < 0.001), education (middle vs low: beta = −0.16, p = 0.008; high vs low: beta = −0.14, p = 0.027) and neurological disability (middle vs low: beta = −0.04, p = 0.441; high vs low: beta = −0.17, p = 0.004) independently predicted MSNQ-P scores (R² = 0.49). In terms of cognitive domains, visuospatial memory (beta = −0.12, p = 0.026) and attention (beta = −0.24, p < 0.001) predicted MSNQ-P scores (R² = 0.08).

Discussion

The optimal MSNQ-P cut-off scores to identify cognitive impairment had insufficient psychometric properties, which is in line with most previous studies.^5–7 Our findings do indicate that certain score ranges are clinically relevant. High MSNQ-P scores (≥27) indicated mild or severe cognitive impairment in 71% of the cases, and we advise neuropsychological evaluation when MS patients obtain these scores. Very low MSNQ-P scores (≤5) indicated intact cognitive function, but we consider this range too small to be clinically relevant. Interestingly, MSNQ-P scores were a better reflection of cognitive status among patients with a higher education than among patients with a middle or lower education. For patients with a higher education, MSNQ-P scores below 12 indicated intact cognitive function. Thus, the MSNQ-P can be clinically useful when interpreting low or high score ranges.

Multiple factors influenced patient-reported cognitive complaints, which may explain the difficulty of defining specific and sensitive cut-off scores. Cognitive function itself, but also depression, anxiety, fatigue, education and neurological disability predicted MSNQ-P scores. Benedict et al. argued that the MSNQ-P might be a useful screening tool when combined with a depression questionnaire.⁴ However, the ability of the MSNQ-P to screen for cognitive impairment did not improve when we excluded patients with a depressed mood (data not shown). In general, the clinical applicability did not improve when either depression, anxiety or fatigue was taken into account.

A promising line of research focuses on short neuropsychological tests as a cognitive screening tool (e.g. the Symbol Digit Modalities Test),¹⁰ which may reflect cognitive impairment more accurately. However, the need for guided assistance in administration could challenge implementation in routine neurological care.

To conclude, MSNQ-P scores in the lowest and highest ranges can indicate whether patients are cognitively intact or impaired. Still, patient-reported cognitive complaints assessed by the MSNQ-P should be interpreted with caution.