Psychometric properties of the Italian version of the Scales for Outcomes in Parkinson’s disease-Cognition (SCOPA-Cog)

Summary

The Scales for Outcomes in Parkinson’s disease-Cognition (SCOPA-Cog) has been shown to be a clinimetrically rigorous and valid instrument for a disease-oriented neuropsychological assessment of Parkinson’s disease (PD) patients. In the present study we evaluated the psychometric properties of the Italian version of the SCOPA-Cog in 121 PD patients. The scale explores memory, attention, and executive and visuospatial functions and takes approximately 20 minutes to administer. Data distribution (skewness= −0.23) and internal consistency (Cronbach’s alpha= 0.78) were satisfactory. Standard error of measurement was 3.42. The outcome was significantly worse in patients with an abnormal score on the Dementia Rating Scale (DRS) (SCOPACog mean score 14.6±5.1 out of a total of 43) with respect to cognitively intact subjects (24.2±4.3) (p<0.0001). The DRS showed good convergent validity (Spearman rho= 0.77, p<0.0001), and a high coefficient of variation (= 0.34). These findings support the goodness of the Italian SCOPA-Cog in terms of metrics and validity.

Introduction

Cognitive impairment is a highly prevalent yet underappreciated non-motor manifestation of Parkinson’s disease (PD), which has a significant impact on patients’ quality of life, prognosis and therapeutic management (Chaudhuri et al., 2011). One main reason why subtle or even more definite neuropsychological deficits may go undetected in patients with PD is the lack of accurate screening and diagnostic tools.

In the past two decades, several tests focusing specifically on the memory-executive-visuospatial profile of PD-associated cognitive impairment have been developed (for a review see Kulisevsky and Pagonabarraga, 2009). One of these specific tools is included in the Scales for Outcomes in Parkinson’s disease (SCOPA) as ‘SCOPA-Cognition’ (SCOPA-Cog) (Marinus et al., 2003). The four neuropsychological domains assessed by the ten subtests included in the SCOPA-Cog are those typically affected in cognitively impaired PD patients: attention, memory, executive functions, and visuospatial abilities. Initially, the scale was developed with the aim of providing a valid instrument for the cognitive assessment of patients with PD for research purposes. However, evidence regarding the discriminative ability of the test also supports its potential as a tool for the identification of PD-associated dementia in the clinical setting. In particular, Dalrymple-Alford et al. (2010) and Verbaan et al. (2011) demonstrated the validity of the instrument in correctly classifying PD patients diagnosed with dementia (PDD) according to the Movement Disorders Society criteria for PDD. In the study by Dalrymple-Alford et al. (2010), the SCOPA-Cog performed on a par with the Montreal Cognitive Assessment (Nasreddine et al., 2005) in identifying PDD. Its diagnostic performance in PD-associated mild cognitive impairment (MCI) was, instead, less satisfactory, due to low specificity and positive predictive values.

Extensive and rigorous evaluation of the psychometric attributes of the original scale has been performed in various studies, yielding generally satisfactory results with regard to score distribution, internal consistency, reliability and validity (Forjaz et al., 2010; Marinus et al., 2003; Serrano-Duenas et al., 2010; Verbaan et al., 2007).

In the present multicenter study we evaluated the clinimetric properties of the Italian version of the SCOPA-Cog in a large population of idiopathic PD patients with and without cognitive impairment. Our aim was to provide movement and cognitive disorders specialists with a metrically sound and valid tool for the neuropsychological assessment of Italian-speaking patients with PD, and to contribute to the international literature on the SCOPA-Cog.

Materials and methods

Study participants were recruited consecutively between January 2007 and December 2010 from the movement disorders clinics of the S. Gerardo Hospital, Monza, Parkinson Institute, Milan, Viareggio Local Health Authority (Azienda Sanitaria Locale), and the Campo di Marte Hospital, Lucca.

They all had a clinical diagnosis of idiopathic PD according to the UK Parkinson’s Disease Society Brain Bank criteria (Gelb et al., 1999). Exclusion criteria were brain injury, serious medical illness, psychiatric disorders, substance abuse, sensory deficits, and moderate to severe depression [score >10 on the 15-item Geriatric Depression Scale, GDS (Sheik and Yesavage, 1986)]. All the subjects were unpaid volunteers and signed a written informed consent form prior to participation. The study was approved by each institution’s ethics committee and conducted in compliance with the principles of the Declaration of Helsinki. Neurological examination was carried out and rated using the motor section of the Unified Parkinson’s Disease Rating Scale (UPDRS), performed with patients in ‘on’ state, and the modified Hoehn & Yahr (H&Y) staging. The neuropsychological protocol included the Mini-Mental Parkinson test (MMP) (Mahieux et al., 1995), the Dementia Rating Scale (DRS) (Mattis, 1988) and the GDS. All neuropsychological tests were performed in a single session, again with the patient in ‘on’ state. A total of five examiners (one per participating center) were involved in test administration and scoring, but three of them contributed 85% of the assessments.

The SCOPA-Cog includes 10 subtests subdivided into four sections: memory and learning, attention, executive functions, and visuospatial functions. Learning and memory are assessed with immediate and delayed recall of a word list presented in written form, and with digit span backward and spatial span. The attention subtests are counting backwards and months backwards (naming the months of the year in reverse order). Executive deficits are evaluated with alternate hand movements, animal fluency and a set-shifting verbal subtest (the subject has to shift between ‘yes’ and ‘no’ answers in response to numerical stimuli). Visuospatial ability is measured through completion of unfinished geometrical patterns. The total score ranges from 0 to 43, with higher scores indicating better performances. The administration time is 15 to 20 minutes.

The forward translation of the original SCOPA-Cog was done by one of the authors (I.M.A.) and compared with the backward translation by a native English speaker. Amendments were made in order to obtain a linguistically and conceptually equivalent Italian version of the scale.

The following psychometric parameters were measured: data acceptability, internal consistency, reliability, convergent validity and validity for known groups. Data acceptability was evaluated considering missing values (acceptable if < 5%) (Lamping et al., 2002), frequency of scores, distance of average score from the median, floor and ceiling effects (acceptable if present in < 15% of cases), and skewness (limits: −0.8 to +0.8) (Hobart et al., 2000; Van der Linden et al., 2005).

Cronbach’s alpha and item-total correlations were computed as indices of internal consistency. Values ≥ 0.70 and ≥ 0.30, respectively, were considered acceptable (Nunnally et al., 1994; Scientific Advisory Committee of the Medical Outcomes Trust, 2002).

Reliability was assessed through precision analysis: the standard error of measurement (SEM) was calculated with the following formula: SD*√(1-reliability index), where reliability index is equivalent to Cronbach’s alpha.

Spearman’s rho statistic was used to establish convergent validity with the DRS. Values ≥ 0.60 are deemed to indicate a strong correlation between the two measures, values from 0.30 to 0.59 indicate a moderate correlation, and values < 0.30 indicate a low correlation (Fisk et al., 2005). Validity for known groups was assessed by comparing SCOPA-Cog scores of PD patients grouped by DRS scores. The cut off was ≥ 126, corresponding to two standard deviations below the average score obtained in a sample of 76 age- and education-matched healthy subjects from the ongoing normative study of the Italian version of the DRS. Finally, discriminant validity was measured as coefficient of variation, corresponding to the standard deviation divided by the mean score obtained on the SCOPA-Cog in the whole study sample. Statistical analysis was performed using PASW statistics 18.0.0 (SPSS Inc., 2009, Chicago, IL, www.spss.com). Student’s t-test, analysis of variance or chi-square analysis were used to compare means of discrete and continuous variables, with a two-tailed standard significance level set at p<0.05. Correlation analysis was carried out using Pearson’s r coefficient or Spearman’s rho coefficient as appropriate.

Results

One-hundred forty-one patients met the inclusion criteria, but three could not be considered because of consent withdrawal and 17 because of incomplete neurological data. The final study sample thus included 121 patients, whose sociodemographic and clinical features are shown in table I.

Table I.

Sociodemographic and clinical features of the study sample.

	Total sample n=121	Subgroup with a DRS score ≥ 126 n=68	Subgroup with a DRS score < 126 n=53
Age	69.7±7.8 (50–86)	66.7±7.5* (50–86)	73.6±6.2 (52–85)
Education	7.9±4.0 (3–17)	9.4±4.0* (5–17)	6.0±3.1 (3–17)
Sex (% of men)	55.4	51.5	60.4
Disease duration	5.9±4.3 (0.2–21.0)	5.4±3.7 (0.2–15)	6.4±4.9 (1.0–21.0)
UPDRS motor score	21.2±11.3 (4–54)	16.7±8.2* (4–39)	26.3±12.1 (4–54)
H&Y stage < 2.5)	62.8	71.9*	51.0
MMP score	25.6±5.1 (7–32)	28.3±2.8* (17–32)	22.1±5.3 (7–32)
DRS score	124.2±13.0 (76–144)	133.6±5.3* (126–144)	112.2±9.5 (76–125)
GDS – short form score	3.3±2.4 (0–9)	3.1±2.2 (0–8)	3.5±2.7 (0–9)

Values are expressed as mean ± standard deviation (range)

^*p<0.0001 vs the subgroup with a DRS score < 126.

Abbreviations: DRS=Dementia Rating Scale; PD=Parkinson’s disease; UPDRS=Unified Parkinson’s Disease Rating Scale; H&Y= Hoehn and Yahr; MMP=Mini-Mental Parkinson test, GDS=Geriatric Depression Scale

Similar sociodemographic and clinical profiles were found between the cases enrolled at the different institutes participating in the study.

The SCOPA-Cog mean score was 20.0±6.7.

There were no missing data. Three scores occurred with the highest frequency: 17, 20 and 24 all had a prevalence of 7.4% (n. of cases: 9). The mean and the median were overlapping (at score 20). The distribution of scores is shown in figure 1; skewness was −0.23. There was no floor or ceiling effect, as no patient obtained the lowest or highest score.

Figure 1.

Histogram showing the distribution of SCOPA-Cog scores in the study population.

Cronbach’s alpha was 0.78. Corrected item-total correlations (Table II) ranged from 0.598 to 0.686. No subtest increased the alpha if deleted.

Table II.

Internal consistency values obtained for the four sections of the SCOPA-Cog.

	Corrected item-total correlation	Cronbach’s alpha if item deleted
Memory and learning	0.685	0.693
Attention	0.598	0.757
Executive functions	0.643	0.696
Visuospatial functions	0.686	0.721

The SEM was 3.42 and the coefficient of variation was 0.34.

Spearman’s rho correlation coefficient was 0.77 between the SCOPA-Cog and both the MMP and the DRS (p<0.0001). The SCOPA-Cog score was also significantly correlated with age (Pearson’s r −0.48, p<0.0001), education (r 0.54, p<0.0001), motor UPDRS (r −0.43, p<0.0001), and disease duration (r −0.22, p<0.01).

The two subgroups of patients divided according to performance on the DRS were not comparable in terms of sociodemographic and neurological characteristics. Fifty-three of the 121 patients (43.8%) obtained a DRS score < 126. Compared with the 68 subjects with a DRS score ≥ 126, they were older, less educated, and more severely impaired on the motor UPDRS and H&Y scale and MMP (Table I). Due to these differences in sociodemographic and neurological characteristics, the SCOPA-Cog scores of the two subgroups were compared with analysis of covariance, using age, education and UPDRS motor score as covariates. Patients with an abnormal DRS performed significantly worse on the scale (14.6±5.1) than the cognitively intact subjects (24.2±4.3) (p<0.0001).

Discussion

A neuropsychological test, to be reliable, must show adequate clinimetric properties. In the present study we investigated the psychometrics of the Italian version of the SCOPA-Cog in a large sample of patients with PD, in order to establish whether the instrument might be proposed to neurologists and neuropsychologists needing a disease-oriented instrument for the cognitive assessment of Italian patients with PD.

In agreement with previous literature (Carod-Artal et al., 2008; Marinus et al., 2003; Serrano-Duenas et al., 2010; Verbaan et al., 2007), we showed that the SCOPA-Cog was able to accurately represent the heterogeneous composition of our neurologically and cognitively mixed study population. Indeed, the scores obtained by our patients covered almost the entire score range of the scale, the distribution was virtually symmetric, the average score overlapped with the median, and there was no ceiling or floor effect. This suggests that the test is adequate for a fine-grained discrimination of patients belonging to an unselected population like ones attending movement disorder and memory clinics.

In the present study we also confirmed previous evidence about the goodness of the instrument’s structure (Carod-Artal et al., 2008; Marinus et al., 2003; Serrano-Duenas et al., 2010), as all the subtests appeared to be tapping the same general cognitive construct, each one being necessary for the consistency of the scale.

Precision analysis yielded less satisfying results, indicating that the difference between true cognitive level and observed SCOPA-Cog scores may be relatively large. Similarly high SEM values were also found in previous series (Carod-Artal et al., 2008; Serrano-Duenas et al., 2010), and call for caution before interpreting intra-individual fluctuations of SCOPA-Cog scores as clinically relevant. In line with the findings of previous investigations (Carod-Artal et al., 2008; Marinus et al., 2003), we found an (expected) significant relationship between the test and age and education. Correction of scores for these factors will probably increase the measurement accuracy of the scale.

Our correlation analysis also revealed significant associations between the SCOPA-Cog and disease duration and UPDRS score, again in line with previous studies (Carod-Artal et al., 2008; Marinus et al., 2003). Further analysis would be needed to clarify whether, or to what extent, these correlations might depend on the influence of motor disability on subtest execution. However, we reckon that this influence is small as only two of the scale’s subtests, accounting for 9 points out of 43, could possibly be affected by ideomotor slowing or akinesia, i.e. verbal fluency and alternate movements.

Finally, our results replicated previous positive findings about the validity of the SCOPA-Cog (Carod-Artal et al., 2008; Marinus et al., 2003; Serrano-Duenas et al., 2010; Verbaan et al., 2007). First of all, the test seemed to be measuring the same construct as the DRS, which can be considered a gold standard for neuropsychological assessment in extrapyramidal diseases (Kulisevsky et al., 2009), and it was able to distinguish patients grouped by their performance on the DRS. Moreover, its coefficient of variation was high, reflecting an ability to detect even minor differences in the degree of cognitive deficits within our sample. On the basis of these results the SCOPA-Cog does seem to efficiently measure the domains in which PD patients typically exhibit impairment, i.e. attention, memory, and executive and visuospatial functions.

In short, our data suggest that the Italian SCOPA-Cog maintains the good metric profile of the original scale, and therefore represents a good option for neuropsychologists who need an instrument allowing an explorative assessment of cognition in Italian individuals affected by PD. A word of caution is needed only with regard to younger patients and patients with moderate to severe depression, who were under-represented in our sample. Attention is also warranted when using the scale for reassessments, or when comparing the outcome obtained by different examiners, given that test-retest and inter-rater reliability were not explored in the present study. Future investigations should also focus on further verifying the diagnostic power of the Italian SCOPA-Cog, in PDD as well as in PD-MCI.